Information theory

Basic concepts of this theory are introduced in the forties of the twentieth century, K. Shannon. Information theory of – the science of the processes of transmission, storage and retrieval of data in natural, technical and social systems. In many applied scientific fields such as computer science, linguistics, cryptography, control theory, image processing, genetics, psychology, Economics, production organization, uses the methods of this science.

In modern conditions the theory of information is very tightly linked to coding theory, addressing the most common problems of conformity between signal and message, and the theory of signal processing in which we study the quantization and reconstruction of signals, spectral and correlation analysis thereof.

Information Theory considers the basic notion of the “information” for the most part, quantitative part, without considering its value, and sometimes the meaning. In this approach, a page of text will contain approximately the same amount of data that is determined only by the number of characters and symbols, and does not depend on the fact that, strictly speaking, there is printed, even though completely meaningless and chaotic a collection of some of the characters.

This approach is valid, but only for modeling communication systems, as they must accurately transmit information via communication channels, which can be any set of characters and symbols. Then, when it is necessary to consider the value and meaning of data, the quantitative approach is unacceptable. Such circumstances impose significant limitations on areas of possible applications of this theory.

The Basics of information theory involve consideration of various issues, including those directly related to the transmission and reception of data. The main scheme of communication, which is seen in the doctrine, as follows. Information theory and coding Believes that the data are generated by a message source, which is a word or set of words, written letters of a certain alphabet. The source of the message can be text in any natural or artificial languages, human speech, a database and a mathematical model that produces a sequence of letters. The transmitter converts the message into a signal corresponding to the physical nature of the communication channel-the medium for signal transmission. While passing itself can be influenced by interference introduces distortions in the values of the information parameters. The receiver recovers the original message from the received distortion signal. In the restored message comes to the addressee-a person or a technical device.

The message Source has a statistical character, i.e. the appearance of each message is determined by a certain probability. Information theory Shannon believes that if the probability of the message is equal to one, i.e. his appearance significantly and uncertainty is not, I think that it conveys no information.

One of the important problems of information theory considers the harmonization of the communication channel and information source properties message. The throughput is the unit in 1 bit per second.

One of the problems of communication systems is interference in the path of the useful signal. Shannon's theorem unfortunately does not give us a specific method of dealing with such. The easiest method of elimination, which consists in repeated repetition of messages is not very efficient, as it requires large amount of time to transfer information. Greater efficiency makes the use of codes that detect and correct transmission errors of information.