Skip to content

Commit 4502cc4

Browse files
authored
Merge pull request #287 from JuliaText/fix/doc_improvements
Fix/doc improvements
2 parents 06c0439 + 0dca707 commit 4502cc4

33 files changed

Lines changed: 648 additions & 520 deletions

docs/src/LM.md

Lines changed: 36 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
1-
# Statistical Language Model
1+
# Statistical Language Models
22

3-
**TextAnalysis** provide following different Language Models
3+
**TextAnalysis** provides the following different language models:
44

5-
- **MLE** - Base Ngram model.
6-
- **Lidstone** - Base Ngram model with Lidstone smoothing.
7-
- **Laplace** - Base Ngram language model with Laplace smoothing.
8-
- **WittenBellInterpolated** - Interpolated Version of witten-Bell algorithm.
9-
- **KneserNeyInterpolated** - Interpolated version of Kneser -Ney smoothing.
5+
- **MLE** - Base n-gram model using Maximum Likelihood Estimation.
6+
- **Lidstone** - Base n-gram model with Lidstone smoothing.
7+
- **Laplace** - Base n-gram language model with Laplace smoothing.
8+
- **WittenBellInterpolated** - Interpolated version of the Witten-Bell algorithm.
9+
- **KneserNeyInterpolated** - Interpolated version of Kneser-Ney smoothing.
1010

1111
## APIs
1212

13-
To use the API, we first *Instantiate* desired model and then load it with train set
13+
To use the API, first instantiate the desired model and then train it with a training set:
1414

1515
```julia
1616
MLE(word::Vector{T}, unk_cutoff=1, unk_label="<unk>") where { T <: AbstractString}
@@ -25,31 +25,31 @@ KneserNeyInterpolated(word::Vector{T}, discount:: Float64=0.1, unk_cutoff=1, unk
2525

2626
(lm::<Languagemodel>)(text, min::Integer, max::Integer)
2727
```
28-
Arguments:
28+
**Arguments:**
2929

30-
* `word` : Array of strings to store vocabulary.
30+
* `word`: Array of strings to store the vocabulary.
3131

3232
* `unk_cutoff`: Tokens with counts greater than or equal to the cutoff value will be considered part of the vocabulary.
3333

34-
* `unk_label`: token for unknown labels
34+
* `unk_label`: Token for unknown labels.
3535

36-
* `gamma`: smoothing argument gamma
36+
* `gamma`: Smoothing parameter gamma.
3737

38-
* `discount`: discounting factor for `KneserNeyInterpolated`
38+
* `discount`: Discounting factor for `KneserNeyInterpolated`.
3939

40-
for more information see docstrings of vocabulary
40+
For more information, see the docstrings of the vocabulary functions.
4141

4242
```julia
4343
julia> voc = ["my","name","is","salman","khan","and","he","is","shahrukh","Khan"]
4444

4545
julia> train = ["khan","is","my","good", "friend","and","He","is","my","brother"]
46-
# voc and train are used to train vocabulary and model respectively
46+
# voc and train are used to train the vocabulary and model respectively
4747

4848
julia> model = MLE(voc)
4949
MLE(Vocabulary(Dict("khan"=>1,"name"=>1,"<unk>"=>1,"salman"=>1,"is"=>2,"Khan"=>1,"my"=>1,"he"=>1,"shahrukh"=>1,"and"=>1), 1, "<unk>", ["my", "name", "is", "salman", "khan", "and", "he", "is", "shahrukh", "Khan", "<unk>"]))
5050

5151
julia> print(voc)
52-
11-element Array{String,1}:
52+
11-element Vector{String}:
5353
"my"
5454
"name"
5555
"is"
@@ -62,42 +62,41 @@ julia> print(voc)
6262
"Khan"
6363
"<unk>"
6464

65-
# you can see "<unk>" token is added to voc
66-
julia> fit = model(train,2,2) #considering only bigrams
65+
# You can see the "<unk>" token is added to voc
66+
julia> fit = model(train,2,2) # considering only bigrams
6767

68-
julia> unmaskedscore = score(model, fit, "is" ,"<unk>") #score output P(word | context) without replacing context word with "<unk>"
68+
julia> unmaskedscore = score(model, fit, "is" ,"<unk>") # score output P(word | context) without replacing context word with "<unk>"
6969
0.3333333333333333
7070

7171
julia> masked_score = maskedscore(model,fit,"is","alien")
7272
0.3333333333333333
73-
#as expected maskedscore is equivalent to unmaskedscore with context replaced with "<unk>"
73+
# As expected, maskedscore is equivalent to unmaskedscore with context replaced with "<unk>"
7474

7575
```
7676

7777
!!! note
7878

79-
When you call `MLE(voc)` for the first time, It will update your vocabulary set as well.
79+
When you call `MLE(voc)` for the first time, it will update your vocabulary set as well.
8080

81-
## Evaluation Method
81+
## Evaluation Methods
8282

8383
### `score`
8484

85-
used to evaluate the probability of word given context (*P(word | context)*)
85+
Used to evaluate the probability of a word given its context (*P(word | context)*):
8686

8787
```@docs
8888
score
8989
```
9090

91-
Arguments:
91+
**Arguments:**
9292

93-
1. `m` : Instance of `Langmodel` struct.
94-
2. `temp_lm`: output of function call of instance of `Langmodel`.
95-
3. `word`: string of word
96-
4. `context`: context of given word
93+
1. `m`: Instance of `Langmodel` struct.
94+
2. `temp_lm`: Output of function call of instance of `Langmodel`.
95+
3. `word`: String of the word.
96+
4. `context`: Context of the given word.
9797

98-
- In case of `Lidstone` and `Laplace` it apply smoothing and,
99-
100-
- In Interpolated language model, provide `Kneserney` and `WittenBell` smoothing
98+
- For `Lidstone` and `Laplace` models, smoothing is applied.
99+
- For interpolated language models, `KneserNey` and `WittenBell` smoothing are provided.
101100

102101
### `maskedscore`
103102
```@docs
@@ -121,28 +120,28 @@ entropy
121120
perplexity
122121
```
123122

124-
## Preprocessing
123+
## Preprocessing
125124

126-
For Preprocessing following functions:
125+
The following functions are available for preprocessing:
127126
```@docs
128127
everygram
129128
padding_ngram
130129
```
131130

132131
## Vocabulary
133132

134-
Struct to store Language models vocabulary
133+
A struct to store language model vocabulary.
135134

136-
checking membership and filters items by comparing their counts to a cutoff value
135+
It checks membership and filters items by comparing their counts to a cutoff value.
137136

138-
It also Adds a special "unknown" tokens which unseen words are mapped to
137+
It also adds a special "unknown" token which unseen words are mapped to:
139138

140139
```@repl
141140
using TextAnalysis
142141
words = ["a", "c", "-", "d", "c", "a", "b", "r", "a", "c", "d"]
143142
vocabulary = Vocabulary(words, 2)
144143
145-
# lookup a sequence or words in the vocabulary
144+
# Look up a sequence of words in the vocabulary
146145
147146
word = ["a", "-", "d", "c", "a"]
148147

docs/src/classify.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,25 @@
11
# Classifier
22

3-
Text Analysis currently offers a Naive Bayes Classifier for text classification.
3+
TextAnalysis currently offers a Naive Bayes Classifier for text classification.
44

5-
To load the Naive Bayes Classifier, use the following command -
5+
To load the Naive Bayes Classifier, use the following command:
66

77
using TextAnalysis: NaiveBayesClassifier, fit!, predict
88

99
## Basic Usage
1010

11-
Its usage can be done in the following 3 steps.
11+
It can be used in the following 3 steps:
1212

13-
1- Create an instance of the Naive Bayes Classifier model -
13+
1. Create an instance of the Naive Bayes Classifier model:
1414
```@docs
1515
NaiveBayesClassifier
1616
```
1717

18-
2- Fitting the model weights on input -
18+
2. Fit the model weights on training data:
1919
```@docs
2020
fit!
2121
```
22-
3- Predicting for the input case -
22+
3. Make predictions on new data:
2323
```@docs
2424
predict
2525
```

0 commit comments

Comments
 (0)