-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathot_anki.Rmd
More file actions
55 lines (37 loc) · 7.35 KB
/
Copy pathot_anki.Rmd
File metadata and controls
55 lines (37 loc) · 7.35 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
---
title: "Knowledge snippets"
output: html_document
---
<br>
Miscellaneous knowledge snippets.
<br>
## Modelling
<br>
| Question | Answer | Reference |
|:------------------------|:------------------|:---------------------------|
| Naive Bayes assumptions | Independence among predictors - assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature. | |
| Which modeling techniques require predictors to have a common scale? | 1\. Neural networks<br/>2. K means clustering<br/>3. SVM's<br/>4. K Nearest Neighbours<br/>5. Any technique using regularisation | |
| What is gradient descent? | Gradient Descent is an optimization algorithm for finding a local minimum of a differentiable function. Gradient descent is simply used to find the values of a function's parameters (coefficients) that minimize a cost function as far as possible. | [Wikipedia](https://en.wikipedia.org/wiki/Gradient_descent) |
| What is regularization? | Regularization is adding a penalty term to the objective function to control the model complexity using that penalty term. When a model overfits data or the predictors are collinear, parameter estimates can become inflated. Adding a penalty stops this and can result in a lower error. | |
| What are regularisation techniques for neural networks? | 1\. Learning rate shrinkage<br>2. Early stopping<br>3. Batch normalization<br>4. Ensembles | Empirical Asset Pricing via Machine Learning(<https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3159577>) provides a good summary |
<br>
## Preprocessing
<br>
| Question | Answer | Reference |
|:------------------------|:------------------|:---------------------------|
| What is polynomial contrast? | The process of transforming ordered categorical variables / predictors to numeric quantities. | [SO 105115](https://stats.stackexchange.com/questions/105115/polynomial-contrasts-for-regression) |
| What characteristics of a response variable indicates it should be transformed prior to modelling? | Responses that have a distribution where the frequency of response proportionally decreases with larger values may indicate that the response follows a log-normal distribution. In this case, log-transforming the response would induce a normal (bell-shaped, symmetric) distribution and often will enable a model to have better predictive performance. | FES^1^ s.4.2.1 |
| xxxxx | xxxxx | xxxxx |
<br>
## Feature selection
<br>
| Question | Answer | Reference |
|:------------------------|:------------------|:---------------------------|
| What is the primary purpose of feature selection? | Removal of non-informative or redundant predictors | FES^1^ |
| What type of models feature automatic feature selection? | 1\. Tree-based models<br/>2. MARS<br/>3. Elastic net / LASSO<br/>4. Nearest shrunken centroids<br/>5. GAM's (use parameter "select = TRUE", effectively the same as LASSO) | [gam.selection](https://stat.ethz.ch/R-manual/R-devel/library/mgcv/html/gam.selection.html) |
| xxxxx | xxxxx | xxxxx |
| xxxxx | xxxxx | xxxxx |
<br>
## Reference
^1^FES : [Feature Engineering and Selection: A Practical Approach for Predictive Models](http://www.feat.engineering/index.html)
Source material : [Google sheets](https://docs.google.com/spreadsheets/d/16ZWgXmCtMD_DHhIUkp0JoA3vrlphjRjO_17dbtWRoUI/edit#gid=0)