# Is data science just glorified data analysis

## Use of information theory in applied data science

Today I came across the book "Information Theory: An Introduction to a Tutorial" by James Stone and had a moment or two about that **Extent of use** of information theory in the **applied ***Data science* pondered (if you are not familiar with this somewhat fuzzy term, think about *Data analysis* , of which IMHO Data Science is a glorified version. I am making significant use of it *Information theory* based *Approaches* , *Methods* and *activities* , especially the *entropy* , **under the hood** aware of various statistical techniques and data analysis methods.

However, I am looking forward to that **Scope /** Level of knowledge a *applied social scientist* needed **, ****around** these concepts, measures and tools are successful **to select** and **apply** without delving too deeply into the mathematical origins of the theory. I look forward to your answers, which address my concerns in connection with the above book (or other similar books - you are welcome to recommend it) or in general.

I would also appreciate some recommendations for print or online sources that support the **Information theory** and their concepts, approaches, methods and measures in **context** (in the **comparison** to) others (rather) **traditional statistical approaches** ( *Frequentist* and *Bayesian* ) to discuss .

Reply:

The first part of the question: *Data scientists need to know information theory* ? I thought the answer was no until recently. The reason I changed my mind is because of a critical component: noise.

Many machine learning models (both stochastic and non-stochastic) use noise as part of their coding and transformation process. In many of these models, you need to infer the probability that the noise will affect after decoding the transformed output of the model. I think this is a central part of information theory. Not only is the KL divergence a very important measure in deep learning, which also comes from information theory.

Second part of the question: I think the best source is David MacKay's Algorithms for Information Theory, Inference, and Learning. He starts with information theory and incorporates these ideas into both inference and neural networks. The PDF is free on Dave's website and the lectures are online which is great

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from.

By continuing, you consent to our use of cookies and other tracking technologies and affirm you're at least 16 years old or have consent from a parent or guardian.

You can read details in our Cookie policy and Privacy policy.

- How do you reduce discrimination against conservatives
- Why is Trump confident about the border
- Why don't marijuana pharmacies use gift cards
- What first got you into chiptune music?
- Why do Sphynx cats leave brown spots
- How can I download the MATLAB software
- Why are children being taught so slowly today
- Can love between couples who will last a lifetime
- Which IT projects can I do
- What is a software repository
- Who should I never seek advice from?
- What are examples of security procedures
- Which doctor are you most afraid of
- Is it a crime to harm society?
- Where does a symmetrical matrix occur naturally?
- What skills must a great DJ master have
- Is the traveler's diarrhea contagious
- Unit 5 is good for game development
- Were Buddhists persecuted by Brahmins?
- What's your favorite KPOP MV
- Who heads the 15th Finance Commission
- Which party won the Sikkim election in 2019?
- What is the fastest sail-propelled watercraft
- What is the difference between Jehovah and Jesus