Please see manuscript for a long description of the following data.
We will load the example data, and you can use the ? with
the dataset name to learn more about the data.
library(lrd)
data("sentence_data")
head(sentence_data)
#> Sub.ID Trial.ID Sentence
#> 1 1 1 This is a sentence.
#> 2 1 2 Woo more sentences!
#> 3 1 3 This is thing that the participant typed.
#> 4 1 4 This is another example sentence.
#> 5 1 5 Okay this is the final thing that they typed.
#> 6 2 1 This is a sentence.
#> Response Condition
#> 1 This is a sentence a
#> 2 Woo more sentences! a
#> 3 This is the thing that the participant typed. a
#> 4 This is another example sentence. a
#> 5 Okay this is the final thing they typed. a
#> 6 This is sentence. b
#?sentence_dataScoring in lrd is case sensitive, so we will use
tolower() to lower case all correct answers and participant
answers.
You should define the following:
Note that the answer key can be in a separate dataframe, use
something like answer_key$answer for the key argument and
answer_key$id_num for the trial number. Fill in
answer_key with your dataframe name and the column name for
those columns after the $.
sentence_ouptut <-
prop_correct_sentence(data = sentence_data,
responses = "Response",
key = "Sentence",
key.trial = "Trial.ID",
id = "Sub.ID",
id.trial = "Trial.ID",
cutoff = 1,
flag = TRUE,
group.by = "Condition",
token.split = " ")
str(sentence_ouptut)
#> List of 3
#> $ DF_Scored :'data.frame': 30 obs. of 11 variables:
#> ..$ Trial.ID : int [1:30] 1 1 1 1 1 1 2 2 2 2 ...
#> ..$ Sub.ID : int [1:30] 1 2 3 4 5 6 2 3 4 1 ...
#> ..$ Sentence : chr [1:30] "this is a sentence." "this is a sentence." "this is a sentence." "this is a sentence." ...
#> ..$ Responses : chr [1:30] "this is a sentence" "this is sentence" "this is a sentence" "this is a sentence" ...
#> ..$ Condition : chr [1:30] "a" "b" "a" "b" ...
#> ..$ Answer : chr [1:30] "this is a sentence" "this is a sentence" "this is a sentence" "this is a sentence" ...
#> ..$ Proportion.Match: num [1:30] 1 0.75 1 1 1 ...
#> ..$ Shared.Items : chr [1:30] "this is a sentence" "this is sentence" "this is a sentence" "this is a sentence" ...
#> ..$ Corrected.Items : chr [1:30] NA NA NA NA ...
#> ..$ Omitted.Items : chr [1:30] NA "a" NA NA ...
#> ..$ Extra.Items : chr [1:30] NA NA NA NA ...
#> $ DF_Participant:'data.frame': 6 obs. of 5 variables:
#> ..$ Condition : chr [1:6] "a" "b" "a" "b" ...
#> ..$ Sub.ID : int [1:6] 1 2 3 4 5 6
#> ..$ Proportion.Correct : num [1:6] 0.978 0.802 0.832 0.646 0.599 ...
#> ..$ Z.Score.Group : num [1:6] 0.916 1.155 0.151 -0.577 -1.067 ...
#> ..$ Z.Score.Participant: num [1:6, 1] 1.564 0.355 0.559 -0.72 -1.039 ...
#> .. ..- attr(*, "scaled:center")= num 0.75
#> .. ..- attr(*, "scaled:scale")= num 0.145
#> $ DF_Group :'data.frame': 2 obs. of 4 variables:
#> ..$ Condition: chr [1:2] "a" "b"
#> ..$ Mean : num [1:2] 0.803 0.698
#> ..$ SD : num [1:2] 0.1908 0.0903
#> ..$ N : int [1:2] 3 3We can use DF_Scored to see the original dataframe with
our new scoring columns - also to check if our answer key and
participant answers matched up correctly! First, each sentence is
stripped of punctuation and extra white space within the function. The
total number of tokens, as split by token.split are tallied
for calculating Proportion.Match. Then, the tokens are
matched using the Levenshtein distance indicated in cutoff,
as with the cued and free recall functions. The key difference in this
function is how each type of token is handled. The
Shared.Items column includes all the items that were
matched completely with the original answer (i.e., a cutoff
of 0). The tokens not matched in the participant answer are then
compared to the tokens not matched from the answer key to create the
Corrected.Items column. This column indicates answers that
were misspelled but within the cutoff score and were matched to the
answer key (i.e., “th” for the, “ths” for this). The non-matched items
are then separated into Omitted.Items (i.e., items in the
answer key not found in the participant answer), and
Extra.Items (i.e., items found in the participant answer
that were not found in the answer key). The
Proportion.Match is calculated by summing the number of
tokens matched in Shared.Items and
Corrected.Items and dividing by the total number of tokens
in the answer key. The DF_Participant can be used to view a
participant level summary of the data. Last, if a grouping variable is
used, we can use DF_Group to see that output.
#Overall
sentence_ouptut$DF_Scored
#> Trial.ID Sub.ID Sentence
#> 1 1 1 this is a sentence.
#> 2 1 2 this is a sentence.
#> 3 1 3 this is a sentence.
#> 4 1 4 this is a sentence.
#> 5 1 5 this is a sentence.
#> 6 1 6 this is a sentence.
#> 7 2 2 woo more sentences!
#> 8 2 3 woo more sentences!
#> 9 2 4 woo more sentences!
#> 10 2 1 woo more sentences!
#> 11 2 6 woo more sentences!
#> 12 2 5 woo more sentences!
#> 13 3 2 this is thing that the participant typed.
#> 14 3 3 this is thing that the participant typed.
#> 15 3 1 this is thing that the participant typed.
#> 16 3 6 this is thing that the participant typed.
#> 17 3 4 this is thing that the participant typed.
#> 18 3 5 this is thing that the participant typed.
#> 19 4 2 this is another example sentence.
#> 20 4 1 this is another example sentence.
#> 21 4 6 this is another example sentence.
#> 22 4 3 this is another example sentence.
#> 23 4 4 this is another example sentence.
#> 24 4 5 this is another example sentence.
#> 25 5 1 okay this is the final thing that they typed.
#> 26 5 2 okay this is the final thing that they typed.
#> 27 5 3 okay this is the final thing that they typed.
#> 28 5 4 okay this is the final thing that they typed.
#> 29 5 5 okay this is the final thing that they typed.
#> 30 5 6 okay this is the final thing that they typed.
#> Responses Condition
#> 1 this is a sentence a
#> 2 this is sentence b
#> 3 this is a sentence a
#> 4 this is a sentence b
#> 5 this thing is a sentence a
#> 6 this is a sentence b
#> 7 woo more sentences b
#> 8 woo more sentecnes a
#> 9 more sentences b
#> 10 woo more sentences a
#> 11 more sentences b
#> 12 woohoo more sentences a
#> 13 this is the thing b
#> 14 the participant typed this thing a
#> 15 this is the thing that the participant typed a
#> 16 this thing was typed b
#> 17 this thing was typed b
#> 18 this sentence was typed a
#> 19 this is an extra example sentence b
#> 20 this is another example sentence a
#> 21 this is anothr example b
#> 22 tis is another xample sentence a
#> 23 this is anothr example b
#> 24 this is another one a
#> 25 okay this is the final thing they typed a
#> 26 okay this is the final thing they typed b
#> 27 okay this is the last thing they typed a
#> 28 ok this is the last one b
#> 29 this is th final one a
#> 30 ok this is the last one b
#> Answer Proportion.Match
#> 1 this is a sentence 1.0000000
#> 2 this is a sentence 0.7500000
#> 3 this is a sentence 1.0000000
#> 4 this is a sentence 1.0000000
#> 5 this is a sentence 1.0000000
#> 6 this is a sentence 1.0000000
#> 7 woo more sentences 1.0000000
#> 8 woo more sentences 0.6666667
#> 9 woo more sentences 0.6666667
#> 10 woo more sentences 1.0000000
#> 11 woo more sentences 0.6666667
#> 12 woo more sentences 0.6666667
#> 13 this is thing that the participant typed 0.5714286
#> 14 this is thing that the participant typed 0.7142857
#> 15 this is thing that the participant typed 1.0000000
#> 16 this is thing that the participant typed 0.4285714
#> 17 this is thing that the participant typed 0.4285714
#> 18 this is thing that the participant typed 0.2857143
#> 19 this is another example sentence 0.8000000
#> 20 this is another example sentence 1.0000000
#> 21 this is another example sentence 0.8000000
#> 22 this is another example sentence 1.0000000
#> 23 this is another example sentence 0.8000000
#> 24 this is another example sentence 0.6000000
#> 25 okay this is the final thing that they typed 0.8888889
#> 26 okay this is the final thing that they typed 0.8888889
#> 27 okay this is the final thing that they typed 0.7777778
#> 28 okay this is the final thing that they typed 0.3333333
#> 29 okay this is the final thing that they typed 0.4444444
#> 30 okay this is the final thing that they typed 0.3333333
#> Shared.Items Corrected.Items
#> 1 this is a sentence <NA>
#> 2 this is sentence <NA>
#> 3 this is a sentence <NA>
#> 4 this is a sentence <NA>
#> 5 this is a sentence <NA>
#> 6 this is a sentence <NA>
#> 7 woo more sentences <NA>
#> 8 woo more <NA>
#> 9 more sentences <NA>
#> 10 woo more sentences <NA>
#> 11 more sentences <NA>
#> 12 more sentences <NA>
#> 13 this is thing the <NA>
#> 14 this thing the participant typed <NA>
#> 15 this is thing that the participant typed <NA>
#> 16 this thing typed <NA>
#> 17 this thing typed <NA>
#> 18 this typed <NA>
#> 19 this is example sentence <NA>
#> 20 this is another example sentence <NA>
#> 21 this is example anothr
#> 22 is another sentence tis xample
#> 23 this is example anothr
#> 24 this is another <NA>
#> 25 okay this is the final thing they typed <NA>
#> 26 okay this is the final thing they typed <NA>
#> 27 okay this is the thing they typed <NA>
#> 28 this is the <NA>
#> 29 this is final th
#> 30 this is the <NA>
#> Omitted.Items Extra.Items
#> 1 <NA> <NA>
#> 2 a <NA>
#> 3 <NA> <NA>
#> 4 <NA> <NA>
#> 5 <NA> thing
#> 6 <NA> <NA>
#> 7 <NA> <NA>
#> 8 sentences sentecnes
#> 9 woo <NA>
#> 10 <NA> <NA>
#> 11 woo <NA>
#> 12 woo woohoo
#> 13 that participant typed <NA>
#> 14 is that <NA>
#> 15 <NA> <NA>
#> 16 is that the participant was
#> 17 is that the participant was
#> 18 is thing that the participant sentence was
#> 19 another an extra
#> 20 <NA> <NA>
#> 21 sentence <NA>
#> 22 <NA> <NA>
#> 23 sentence <NA>
#> 24 example sentence one
#> 25 that <NA>
#> 26 that <NA>
#> 27 final that last
#> 28 okay final thing that they typed ok last one
#> 29 okay thing that typed one
#> 30 okay final thing that they typed ok last one
#Participant
sentence_ouptut$DF_Participant
#> Condition Sub.ID Proportion.Correct Z.Score.Group Z.Score.Participant
#> 1 a 1 0.9777778 0.9160221 1.5637499
#> 2 b 2 0.8020635 1.1547005 0.3553233
#> 3 a 3 0.8317460 0.1508220 0.5594568
#> 4 b 4 0.6457143 -0.5773503 -0.7199254
#> 5 a 5 0.5993651 -1.0668441 -1.0386793
#> 6 b 6 0.6457143 -0.5773503 -0.7199254
#Groups
sentence_ouptut$DF_Group
#> Condition Mean SD N
#> 1 a 0.8029630 0.19084127 3
#> 2 b 0.6978307 0.09026826 3