Porbabilistic Machine Learning

 Stochastic Variational Inference

https://www.it.uu.se/research/systems_and_control/education/2018/pml/lectures/

Such a good source for probabilistic ML.

Integration: p(D) = INT( p(D |w)p(w)dt ) 

Optimization: D=arg max p(D|w) over w parameter

The three cornerstones: 

1. (Data) The observed data becomes useful when we have extracted knowledge from it. 

2. (Mathematical model) A mathematical model is a compact representation of the data that in precise mathematical form captures the key properties of the underlying situation. 

3. (Learning algorithm) Used to compute the unknown variables from the observed data using the model.

Key probabilistic objects (notation: D - measured data and w - unknown model variables): 

The full probabilistic model (joint distribution of all known and unknown variables present in the model) is given by

 p(D,w) = p(D |w) | {z } data distribution p(D|w) and {z } prior

 In the Bayesian setting learning amounts to computing the posterior distribution 

p(w | D) = [p(D |w) |  p(w)] /p(D) (i.e., p(D|w) likelihood , p(w) prior, p(D|w) marginal likelihood)

1)p(x, z) = p(x | z) | {z } likelihood p(z) |{z} prior

2)Marginal likelihood, p(x) = INT[ p(x, z)dz],  is highly intractable for many models of interest.

1)SVI is also called Stochastic: Markov chain Monte Carlo (MCMC), sequential Monte Carlo (SMC), stochastic variational inference 

2) Deterministic: variational inference, expectation propagation

Physical Modeling with Python

 https://physicalmodelingwithpython.blogspot.com/

Such a good blog.

 Entropy, Relative Entropy, Cross Entropy

https://www.iitg.ac.in/cseweb/osint/slides/Anasua_Entropy.pdf

It is basically coming from Real Analysis of Measure Theory goes beyond Information Theory: Shannon Entropy

https://en.wikipedia.org/wiki/Entropy_(information_theory)#Definition

Lesser the probability for an event, larger the entropy. >>> Entropy of a six-headed fair dice is log2(6).

Machine Learning Notes with R studio

 There are 4 types of machine learning algorithms:

1. Supervised (where the data is labeled outputs are categorical and regression--->numerical values)

2. Unsupervised (unlabeled data and to detect the structure of the input, and categorized as an output)

i.clustering

ii. dimension reduction (NMF, PCA..)

3. Semi-supervised (unsupervised techniques first later supervised methods on the same input)

4. Reinforcement learning (a method that uses feedback from operating in a real or synthetic environment.

Staying Fresh and UpToDate. One more project.. The decision of best rank preprint out on bioRxiv

 it has been too many months I have not posted here. Stay fresh notebook. 

Here you go one more project:

https://www.biorxiv.org/content/10.1101/2022.04.14.488288v1.article-metrics

FPKM vs read counts of RNA-seq data

"A quick example of the technical aspect: 
assume a 1,000 bp transcript. experiment 1 is 5,000,000 total reads and this transcript received 5 bhits. This calculates out to an FPKM of 1.0. But that FPKM is based on only 5 hits which is entirely unreliable. experiment 2 has 100,000,000 total reads and this transcript has 100 hits. This also calculates out to an FPKM of 1.0 however this FPKM is much more reliable as it's based on 100 hits which is a more stable count level. the variance due to aligner error and count methods might only vary that count value by 5% whereas the count of 5 could vary by 80% or more."
 link of the given information: http://seqanswers.com/forums/showthread.php?t=30269

NMF for dummies

http://www.billconnelly.net/?p=534

estimating NMF rank how to choose r<=min(m,n)

this is from NMF package from R studio: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.192.3637&rep=rep1&type=pdf

"Several approaches have been proposed to choose the optimal value of r. For example, [Brunet et al., 2004] proposed to take the first value of r for which the cophenetic coefficient starts decreasing, [Hutchins et al., 2008] suggested to choose the first value where the RSS curve presents an inflection point, and [Frigyesi and H¨oglund, 2008] considered the smallest value at which the decrease in the RSS is lower than the decrease of the RSS obtained from random data. "


another useful link:
https://stackoverflow.com/questions/17199575/explain-extractfeatures-from-the-nmf-package-in-r

PCA vs NMF

1)PCA and NMF optimize for a different result. 

2)PCA finds a new subspace which takes the same variance of the data and leads to a new feature. It is a dimension reduction method.

3)NMF finds nonnegative features of the given data, however one should be careful because NMF is very sensitive  to initialization, and hence won’t find the same features every time.

4)Output of NMF can be visualized as a smaller version of original dataset so that one would not have to deal with bigger dataset.

5) NMF is more useful most of the time Interpretability. The key is that all of the features learned via NMF are additive; that is, every point in the transformed space can be constructed by adding together strictly positive features.  (http://dx.doi.org/10.1109/IJCNN.2004.1381038)

Mixture Models

Gompertz Mixture Model

Nice tutorial: http://documentation.statsoft.com/STATISTICAHelp.aspx?path=Glossary/GlossaryTwo/C/CensoringCensoredObservations

Python-machine learning auto data

Python, pandas and numpy installation
solves the problem

C:\> setx PATH "%PATH%;C:\<path\to\python\folder>\Scripts"

C:\> pip install pandas


I wasted time to find out the correct url link:



url = "https://archive.ics.uci.edu/ml/machine-learning-databases/autos/imports-85.data"

BLAST-Basic Local Alignment Search Tool

Standalone BLAST on Linux machine:

BLAST detects regions of local similarities between sequences.

• How to Create your own database and search for the desired sequence?

1. First way is wget on command line;

which is $   wget ftp://ftp.ncbi.nih.gov/blast/db/FASTA/nr.gz

      $ wget -c ftp://ftp.ncbi.nih.gov/blast/db/FASTA/nr.gz 

-c means to continue

1. NCBI ftp website 

notes on Drummond et al "Why highly expressed proteins evolve slowly" paper

Some stuff:

30 years ago, Zuckerkandl proposed that a protein’s sequence will evolve at a rate primarily determined by the proportion of its sites involved in specific functions (or ‘‘functional density’’).

However, those effects in functional density and measurements of residues in protein functions remained unclear.

gene duplication in bacteria

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2787491/pdf/1745-6150-4-46.pdf

Nice article.

SQL

Adapted from codecademy.com SQL course:

• JOIN will combine rows from different tables if the join condition is true.
• LEFT JOIN will return every row in the left table, and if the join condition is not met, NULL values are used to fill in the columns from the right table.
• Primary key is a column that serves a unique identifier for the rows in the table.
• Foreign key is a column that contains the primary key to another table.
• CROSS JOIN lets us combine all rows of one table with all rows of another table.
• UNION stacks one dataset on top of another.
• WITH allows us to define one or more temporary tables that can be used in the final query.

Learn SQL: Create Table

learn SQL codecademy.com

We learned 6 commands in SQL today;

A statement is a string of characters that the database recognizes as a valid command.

• CREATE TABLE creates a new table.
• INSERT INTO adds a new row to a table.
• SELECT queries data from a table.
• UPDATE edits a row in a table.
• ALTER TABLE changes an existing table.
• DELETE FROM deletes rows from a table

How to crete R Mark Down files in R studio

need to install:
install.package("rmarkdown")

how to render Rmd file through terminal command line after $ssh username

$ echo "rmarkdown::render('sample.Rmd', clean=TRUE)" | R --slave

$ Rscript -e "rmarkdown::render('sample.Rmd', clean=TRUE)”

Need to have "pandoc" package on X11 or Linux machines.
$pandoc -v

solving ODEs

A nice recalling examples here:

http://mathinsight.org/ordinary_differential_equation_introduction

data science tools and machine learning

https://turi.com/learn/userguide/

https://www.coursera.org/learn/ml-foundations/home/welcome