الفهرس Only 14 pages are availabe for public view
 |  | from | 107 |  |  |
| | | from | 107 | | |
Abstract
In many real life applications we are faced with a finite sequence of. letters or symbols from some information source. And in order to investigate the properties of this time-series several methods can be involved and one of the main methods is to investigate the absolute entropy of the source.
Entropy of some source can tell us how informative its output is and how complex the source structure might be.
Many methods were developed by various researchers for estimating the entropy. Depending on the nature of the time-series of symbols and any priori knowledge of the source itself; the method should be chosen. Two main classes exist: Parametric and non-Parametric Entropy estimation.
We will lay out a new technique for non-parametric entropy estimation. This technique investigates the gradient of the entropy estimated using different memory and word sizes for studying the data source.
The technique proposed while investigating the entropy of a source, it extracts a set of results that can be used in various applications. Such applications include; deciding the memory needed to implement the data source (Finite state machines in general), regeneration of the source or a sequence with similar statistical properties, approximating machines using others with less memory, etc. Also, it has been shown that such results can have applications in investigating and testing some properties of graphs, integer numbers and sequences with special forms.
The new technique when applied to some types of data sources gave better results and faster convergence than many other well known techniques.