{"id":274,"date":"2022-05-28T03:07:20","date_gmt":"2022-05-28T03:07:20","guid":{"rendered":"https:\/\/gatesboltonanalytics.com\/?page_id=274"},"modified":"2022-09-17T18:28:48","modified_gmt":"2022-09-17T18:28:48","slug":"naive-bayes-in-r","status":"publish","type":"page","link":"https:\/\/gatesboltonanalytics.com\/?page_id=274","title":{"rendered":"Naive Bayes in R"},"content":{"rendered":"\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<pre class=\"wp-block-code\"><code>#########################################################\n##\n##          Tutorial: Naive Bayes (etc) in R for Text data \n##                    and for Record data. Text will be corpus\n##                    \n##\n##           Topics:\n##             - Tokenization\n##             - Vectorization\n##             - Normalization\n##             - Naive Bayes\n##\n##     THE DATA CORPUS IS HERE: \n##  https:\/\/drive.google.com\/drive\/folders\/1J_8BDiOttPvEYW4-JxrReKGP1wN40ccy?usp=sharing \n##  Also - for fun - here is another corpus to play with\n##  https:\/\/drive.google.com\/drive\/folders\/1NnPZfhg_04lUAdKPNMMqkTcyfF6Z0BP2?usp=sharing\n##\n##\n##     RECORD DATA IS HERE:\n##  https:\/\/drive.google.com\/file\/d\/18dJPOiiO9ogqOibJppc0lsDiQ2-bQs0f\/view?usp=sharing\n#########################################################\n## Gates\n#########################################################\n\n\nlibrary(tm)\n#install.packages(\"tm\")\nlibrary(stringr)\nlibrary(wordcloud)\n# ONCE: install.packages(\"Snowball\")\n## NOTE Snowball is not yet available for R v 3.5.x\n## So I cannot use it  - yet...\n##library(\"Snowball\")\n##set working directory\n## ONCE: install.packages(\"slam\")\nlibrary(slam)\nlibrary(quanteda)\n## ONCE: install.packages(\"quanteda\")\n## Note - this includes SnowballC\nlibrary(SnowballC)\nlibrary(arules)\n##ONCE: install.packages('proxy')\nlibrary(proxy)\nlibrary(cluster)\nlibrary(stringi)\nlibrary(proxy)\nlibrary(Matrix)\nlibrary(tidytext) # convert DTM to DF\nlibrary(plyr) ## for adply\nlibrary(ggplot2)\nlibrary(factoextra) # for fviz\nlibrary(mclust) # for Mclust EM clustering\n\nlibrary(naivebayes)\n#Loading required packages\n#install.packages('tidyverse')\nlibrary(tidyverse)\n#install.packages('ggplot2')\nlibrary(ggplot2)\n#install.packages('caret')\nlibrary(caret)\n#install.packages('caretEnsemble')\nlibrary(caretEnsemble)\n#install.packages('psych')\nlibrary(psych)\n#install.packages('Amelia')\nlibrary(Amelia)\n#install.packages('mice')\nlibrary(mice)\n#install.packages('GGally')\nlibrary(GGally)\nlibrary(e1071)\n\n\n####### USE YOUR OWN PATH ############\nsetwd(\"C:\\\\Users\\\\profa\\\\Documents\\\\RStudioFolder_1\\\\DrGExamples\\\\ANLY501\\\\DATA\\\\\")\n##\n## Raad in the datasets\n## Remember - we have TWO (2) datasets here.\n## One is text data from a corpus called Novels_Corpus (link above)\n## The other is record data as csv - the Summer Student dataset.\n##\n##\nStudentDataFile=\"SummerStudentDataset_Record_Mixed_Labeled.csv\"\nhead(StudentDF&lt;-read.csv(StudentDataFile))\n## Next, load in the documents (the corpus)\nNovelsCorpus &lt;- Corpus(DirSource(\"Novels_Corpus\"))\n(ndocs&lt;-length(NovelsCorpus))\n\n(summary(NovelsCorpus))  ## This will list the docs in the corpus\n\n###################################################################\n#######       Change the COrpus into a DTM, a DF, and  Matrix\n#######\n####################################################################\n\nNovels_dtm &lt;- DocumentTermMatrix(NovelsCorpus,\n                                 control = list(\n                                   #stopwords = TRUE, ## remove normal stopwords\n                                   wordLengths=c(4, 8), ## get rid of words of len 3 or smaller or larger than 15\n                                   removePunctuation = TRUE,\n                                   removeNumbers = TRUE,\n                                   tolower=TRUE,\n                                   #stemming = TRUE,\n                                   remove_separators = TRUE\n                                   #stopwords = MyStopwords,\n                                   \n                                   #removeWords(MyStopwords),\n                                   #bounds = list(global = c(minTermFreq, maxTermFreq))\n                                 ))\n########################################################\n################### Have a look #######################\n################## and create formats #################\n########################################################\n#(inspect(Novels_dtm))  ## This takes a look at a subset - a peak\nDTM_mat &lt;- as.matrix(Novels_dtm)\n(DTM_mat&#91;1:13,1:10])\n\n#########################################################\n######### OK - Pause - now the data is vectorized ######\n## Its current formats are:\n## (1) Novels_dtm is a DocumentTermMatrix R object\n## (2) DTM_mat is a matrix\n#########################################################\n\n#Novels_dtm &lt;- weightTfIdf(Novels_dtm, normalize = TRUE)\n#Novels_dtm &lt;- weightTfIdf(Novels_dtm, normalize = FALSE)\n\n## Look at word freuqncies out of interest\n(WordFreq &lt;- colSums(as.matrix(Novels_dtm)))\n\n(head(WordFreq))\n(length(WordFreq))\nord &lt;- order(WordFreq)\n(WordFreq&#91;head(ord)])\n(WordFreq&#91;tail(ord)])\n## Row Sums\n(Row_Sum_Per_doc &lt;- rowSums((as.matrix(Novels_dtm))))\n\n## I want to divide each element in each row by the sum of the elements\n## in that row. I will test this on a small matrix first to make \n## sure that it is doing what I want. YOU should always test ideas\n## on small cases.\n#############################################################\n########### Creating and testing a small function ###########\n##           \n##                  Practice and test first!\n##\n#############################################################\n## Create a small pretend matrix\n## Using 1 in apply does rows, using a 2 does columns\n(mymat = matrix(1:12,3,4))\nfreqs2 &lt;- apply(mymat, 1, function(i) i\/sum(i))  ## this normalizes\n## Oddly, this re-organizes the matrix - so I need to transpose back\n(t(freqs2))\n## OK - so this works. Now I can use this to control the normalization of\n## my matrix...\n#############################################################\n\n## Copy of a matrix format of the data\nNovels_M &lt;- as.matrix(Novels_dtm)\n(Novels_M&#91;1:13,1:5])\n\n## Normalized Matrix of the data\nNovels_M_N1 &lt;- apply(Novels_M, 1, function(i) round(i\/sum(i),5))\n(Novels_M_N1&#91;1:13,1:5])\n## NOTICE: Applying this function flips the data...see above.\n## So, we need to TRANSPOSE IT (flip it back)  The \"t\" means transpose\nNovels_Matrix_Norm &lt;- t(Novels_M_N1)\n(Novels_Matrix_Norm&#91;1:13,1:10])\n\n############## Always look at what you have created ##################\n## Have a look at the original and the norm to make sure\n(Novels_M&#91;1:13,1:10])\n(Novels_Matrix_Norm&#91;1:13,1:10])\n\n######################### NOTE #####################################\n## WHen you make calculations - always check your work....\n## Sometimes it is better to normalize your own matrix so that\n## YOU have control over the normalization. For example\n## scale used diectly may not work - why?\n\n##################################################################\n###############   Convert to dataframe     #######################\n##################################################################\n\n## It is important to be able to convert between format.\n## Different models require or use different formats.\n## First - you can convert a DTM object into a DF...\n\nNovels_DF &lt;- as.data.frame(as.matrix(Novels_dtm))\n(Novels_DF&#91;1:4, 1:4])\n#(head(Novels_DF))\nstr(Novels_DF)\n(Novels_DF$aunt)\n(nrow(Novels_DF))  ## Each row is a novel\n## Fox DF format\n\n######### Next - you can convert a matrix (or normalized matrix) to a DF\nNovels_DF_From_Matrix_N&lt;-as.data.frame(Novels_Matrix_Norm)\n(Novels_DF_From_Matrix_N&#91;1:4, 1:4])\n\n#######################################################################\n#############   Making Word Clouds ####################################\n#######################################################################\n## This requires a matrix - I will use Novels_M from above. \n## It is NOT mornalized as I want the frequency counts!\n## Let's look at the matrix first\n(Novels_M&#91;c(1:13),c(3850:3900)])\nwordcloud(colnames(Novels_M), Novels_M&#91;13, ], max.words = 100)\n\n############### Look at most frequent words by sorting ###############\n(head(sort(Novels_M&#91;13,], decreasing = TRUE), n=20))\n\n\n\n\n########### Frequencies and Associations ###################\n\n## FInd frequenct words...\n(findFreqTerms(Novels_dtm, 2500))\n## Find assocations with aselected conf\n(findAssocs(Novels_dtm, 'aunt', 0.98))\n\n############## -------------- ############################\n##\n## NAIVE BAYES\n## \n#################################################\n\n## We have a dataframe of text data called Novels_DF_From_Matrix_N\n(Novels_DF&#91;1:5, 1:5])\n\n## Next, because our file names, such as Austen_Emma, etc are the labels, \n## we can use these to create a dataframe of labels and then bind that\n## to our Novels_DF_From_Matrix_N\n\n## Get the row names\n(DF_Row_Names &lt;- row.names(Novels_DF))\n\n## For Naive Bayes, we want all the authors to be labeled - so all Austen books to\n## Austen, all Cbronte books to be CBronte, etc. \n\n\n## New and empty list for the labels\nMyNamesList &lt;- c()\nfor(next_item in DF_Row_Names){\n  #print(next_item)\n  ## \n  Names &lt;- strsplit(next_item, \"_\")\n  Next_Name&lt;- Names&#91;&#91;1]]&#91;1]\n  if(Next_Name == \"EBronte\"){\n    Next_Name&lt;-\"CBronte\"\n  }\n  MyNamesList&lt;-c(MyNamesList,Next_Name)\n}\n\n### Use the list of labels to bind together with your DF to created labeled data.\nprint(MyNamesList)\n\nLabeled_DF_Novels &lt;- cbind(MyNamesList, Novels_DF)\n(Labeled_DF_Novels&#91;1:5, 1:5])\n\n## Then, create a DF with no row names\nrownames(Labeled_DF_Novels) &lt;- c()\n\n## Check both\n(Labeled_DF_Novels&#91;1:5, 1:5])\n\n############ !!!!!!!!!!!!!!!!!!! ##\n## Notice that the NAME OF THE LABEL\n## is \"MyNamesList\"\n########################################\n\n##########################################################\n##\n##  Create the Testing and Training Sets         \n##\n########################################################\n#####################################################\n##        Grabbing Every X value  ##################\n####################################################\n## \n## This method works whether the data is in order or not.\nX = 3   ## This will create a 1\/3, 2\/3 split. \n## Of course, X can be any number.\n(every_X_index&lt;-seq(1,nrow(Labeled_DF_Novels),X))\n\n## Use these X indices to make the Testing and then\n## Training sets:\n\nDF_Test&lt;-Labeled_DF_Novels&#91;every_X_index, ]\nDF_Train&lt;-Labeled_DF_Novels&#91;-every_X_index, ]\n## View the created Test and Train sets\n(DF_Test&#91;1:5, 1:5])\n(DF_Train&#91;1:5, 1:5])\n\n\n## Use tables to check the balance...\n## WARNING - do not do this is you have high D data !!!\n## DO NOT RUN THESE ON HIGH D DATA :)\n##sapply(DF_Test, table)  ## Looks good!\n##sapply(DF_Train, table)  ## Looks good!\n\n################  OK - now we have a Training Set\n###############        and a Testing Set.\n\n\n\n## Remove and KEEP the label from the test &amp; train sets.\n## Make sure label is factor type\nstr(DF_Test$MyNamesList)  ## Notice that the label is called \"MyNamesList\" and\n## is correctly set to type FACTOR. This is IMPORTANT!!\nstr(DF_Train$MyNamesList)  ## GOOD! also type FACTOR\n\n## Copy the Labels\n(DF_Test_Labels &lt;- DF_Test$MyNamesList)\nstr(Test_Labels)\n## Remove the labels\nDF_Test_NL&lt;-DF_Test&#91; , -which(names(DF_Test) %in% c(\"MyNamesList\"))]\n(DF_Test&#91;1:5, 1:5])\n\n## REMOVE THE LABEL FROM THE TRAINING SET...\nDF_Train_Labels&lt;-DF_Train$MyNamesList\nDF_Train_NL&lt;-DF_Train&#91; , -which(names(DF_Train) %in% c(\"MyNamesList\"))]\n(DF_Train_NL&#91;1:5, 1:5])\n\n###########################################################\n##\n##          Prepare the record data for \n##          analysis. Create testing and\n##          training sets.\n##\n##          REMEMBER - we are assuming here that\n##          the record dataset is already 100% clean\n################################################################\n## Record Data..................\n## MAKE test and train data\n(Size &lt;- (as.integer(nrow(StudentDF)\/4)))  ## Test will be 1\/4 of the data\n(SAMPLE &lt;- sample(nrow(StudentDF), Size))\n\n(DF_Test_Student&lt;-StudentDF&#91;SAMPLE, ])\n(DF_Train_Student&lt;-StudentDF&#91;-SAMPLE, ])\n##\n## REMOVE the labels and KEEP THEM\n##   \n## \nstr(DF_Test_Student$Decision)  ## Notice that the label is called \"Decision\" and\n## is correctly set to type FACTOR. This is IMPORTANT!!\nstr(DF_Train_Student$Decision)  ## GOOD! Here \"Decision\" is also type FACTOR\n##Check balance of test dataset\ntable(DF_Test_Student$Decision)\n##################################### REMOVE AND SAVE LABELS...\n## Copy the Labels\n(DF_Test_Student_Labels &lt;- DF_Test_Student$Decision)\n## Remove the labels\nDF_Test_Student_NL&lt;-DF_Test_Student&#91; , -which(names(DF_Test_Student) %in% c(\"Decision\"))]\n(DF_Test_Student_NL&#91;1:5, 1:5])\n## Check size\n(ncol(DF_Test_Student_NL))\n#(DF_Test_Student_NL)\n## Train...--------------------------------\n## Copy the Labels\n(DF_Train_Student_Labels &lt;- DF_Train_Student$Decision)\n## Remove the labels\nDF_Train_Student_NL&lt;-DF_Train_Student&#91; , -which(names(DF_Train_Student) %in% c(\"Decision\"))]\n(DF_Train_Student_NL&#91;1:5, 1:5])\n## Check size\n(ncol(DF_Train_Student_NL))\n#(DF_Train_Student_NL)\n\n###############################################################\n##\n##                         NAIVE BAYES\n##\n###############################################################\n\n############\n## What do we have?\n## ------------------------------------\n## For Text Data, we have:\n## ---------------------------------\n## DF_Test_NL    ## Text testset\n## DF_Train_NL   ## Text Training set\n## The label is called \"MyNamesList\" \n## DF_Test_Labels  ## Testset labels\n## DF_Train_Labels  ## training labels\n##\n## ----------------------------\n## For Record data, we have:\n##-----------------------------\n## DF_Test_Student_NL  ## Testset\n## DF_Train_Student_NL  ## Training set\n## Label name is \"Decision\"\n## Test labels:\n## DF_Test_Student_Labels\n## DF_Train_Student_Labels\n######################################\n\n\n-------------------------------------------------------------\n  \n  ## Just FYI......................if memory or overflow issues....\n  ## memory.limit()\n  #data=DF_Train&#91;,1:5000]\n  #(data&#91;1:5, 1:5])\n  ##\n  \n  #################################################################\n##\n####### RUN Naive Bayes --------------------------------------------\n##\n######################################################################\n\n## TEXT DATA\n## https:\/\/www.rdocumentation.org\/packages\/e1071\/versions\/1.7-3\/topics\/naiveBayes\n##########################################################################\nNB_e1071_2&lt;-naiveBayes(DF_Train_NL, DF_Train_Labels, laplace = 1)\nNB_e1071_Pred &lt;- predict(NB_e1071_2, DF_Test_NL)\n#NB_e1071_2\ntable(NB_e1071_Pred,DF_Test_Labels)\n(NB_e1071_Pred)\n\n\n\n##########################################################\n## Record Data----------------------------\n#######################################################\n(NB_e1071_Student&lt;-naiveBayes(DF_Train_Student_NL, DF_Train_Student_Labels, laplace = 1))\nNB_e1071_Pred_Student &lt;- predict(NB_e1071_Student, DF_Test_Student_NL)\ntable(NB_e1071_Pred_Student,DF_Test_Student_Labels)\n(NB_e1071_Pred_Student)\n\n\n\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"class_list":["post-274","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=\/wp\/v2\/pages\/274","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=274"}],"version-history":[{"count":2,"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=\/wp\/v2\/pages\/274\/revisions"}],"predecessor-version":[{"id":689,"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=\/wp\/v2\/pages\/274\/revisions\/689"}],"wp:attachment":[{"href":"https:\/\/gatesboltonanalytics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=274"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}