{"id":4286,"date":"2018-09-18T18:51:18","date_gmt":"2018-09-18T22:51:18","guid":{"rendered":"https:\/\/www.amyork.ca\/academic\/zz\/?p=4286"},"modified":"2018-09-18T18:57:49","modified_gmt":"2018-09-18T22:57:49","slug":"convergent-and-discriminant-validity-evidence","status":"publish","type":"post","link":"https:\/\/www.amyork.ca\/academic\/zz\/psychological-measurement\/convergent-and-discriminant-validity-evidence\/","title":{"rendered":"Convergent and discriminant validity evidence"},"content":{"rendered":"<p><strong>Convergent and discriminant evidence:<\/strong><\/p>\n<ul>\n<li>Convergent and discriminant validity reflects the degree to which test scores have the<\/li>\n<\/ul>\n<p>\u2018correct\u2019 patterns of associations with other variables<\/p>\n<ul>\n<li>The conceptual foundation of a construct includes the connections between the construct and a variety of other psychological constructs<\/li>\n<li>The interconnections between a construct and other related constructs are known collectively as a nomological network<\/li>\n<\/ul>\n<p><strong>Methods for evaluating convergent and discriminant validity:<\/strong><\/p>\n<ul>\n<li>Focussed association <strong>o <\/strong>Where a particular correlation between test scores and a specified criterion are \u2018make or break\u2019 <strong>o <\/strong>g. SAT scores, it must actually correlate with university marks <strong>o <\/strong>Also known as predictive validity <strong>o <\/strong>Validity generalisation- the correlation is valid across different people, sexes etc.<\/li>\n<li>Sets of correlations <strong>o <\/strong>Focuses on a select few validity coefficients, a nomological network is typically based on a large number of coefficients across a wide range of criteria<\/li>\n<\/ul>\n<p><strong>o <\/strong>rnce the coefficients are estimated, they are placed into a table for visual inspection <strong>o <\/strong>A somewhat subjective judgement is then made about the pattern of coefficients with respect to whether a judgement of construct validity is merited<\/p>\n<ul>\n<li>Multitrait-multimethod matrices:<\/li>\n<\/ul>\n<p><strong>o <\/strong>rvercomes the fact that a correlation between two scores may conflate two sources of variances:<\/p>\n<ul>\n<li>Trait variance (good stuff)<\/li>\n<li>Method variance (bad stuff)<\/li>\n<\/ul>\n<ul>\n<li>Researchers who use the NTMM approach must administer their inventory using different methods (at least three)<\/li>\n<li>Based on the notion that we \u2018hope\u2019 to see larger correlations between the same traits (irrespective of method of measurement), in comparison to correlations between scores based simply on the same method<\/li>\n<li>That is, large correlations between different traits but the same measurement method are not interesting theoretically<\/li>\n<li>Suggests correlations are simply due to a response style (i.e. method variance) <strong>o <\/strong>g. consider a measure of social skill to validate\n<ul>\n<li>Could administer the questionnaire in addition to other measures such as impulsivity, emotional stability<\/li>\n<li>Theoretically expect some \u2018small-ish\u2019 correlations between social skill land these other three measures<\/li>\n<li>This would be the multitrait component of the study <strong>o <\/strong>In addition to collecting data via the self-report method, we could also use the reports from acquaintances.<\/li>\n<\/ul>\n<\/li>\n<li>That is, each person who completed the questionnaires would have someone that knows them well to complete the same questions phrased in the third person<\/li>\n<li>Traits could also be measured using an interview-based technique <strong>o <\/strong>Therefore three methods of measurement: self-report, rater-report, interview <strong>o <\/strong>Evaluating MTMM:\n<ul>\n<li>No clear guidelines to evaluate differences in mean correlations<\/li>\n<li>Very labour intensive<\/li>\n<\/ul>\n<\/li>\n<li>Quantifying construct validity:\n<ul>\n<li>Requires researchers to predict the magnitude of the correlation between their measure of interest and their selected criteria<\/li>\n<li>Then the correlation between their measure of interest and these selected criteria are estimated. Finally, the correlation between the predicted and estimated correlations are estimated.<\/li>\n<li>g. professors guess what the correlation would be between the measure of social motivation and 12 other self-reported personality attributes\n<ul>\n<li>Took averages of the estimates<\/li>\n<li>Then got people to respond to the questionnaires and estimated the empirical correlations<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>o <\/strong>Limitations<\/p>\n<p>\uf0a7 Still unsure how large a sufficient correlation is<\/p>\n<p><strong>Factors affecting a validity coefficient:<\/strong><\/p>\n<ul>\n<li>Magnitude would be effected by a number of factors:\n<ul>\n<li>Some statistical<\/li>\n<li>rthers more measurement related<\/li>\n<\/ul>\n<\/li>\n<li>Measurement error:\n<ul>\n<li>Be biased downwardly to the extent that the measure are associated with scores with imperfect reliability<\/li>\n<li>If your measures have internal consistency reliabilities in the area of .50 to .60 you shouldn\u2019t expect much in the way of validity coefficients<\/li>\n<li>Correlation coefficients can be disattenuated for imperfect reliability using Cronbach\u2019s alpha<\/li>\n<\/ul>\n<\/li>\n<li>Restricted range:\n<ul>\n<li>The amount of variability in one or both distributions of screes can affect the correlation between the two sets of scores<\/li>\n<li>There are two realistic cases where the amount of variability is lower than it should or could be:\n<ul>\n<li>The measures are not sensitive enough to distinguish people<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>The sample is more homogenous than the population of interest<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Convergent and discriminant evidence: Convergent and discriminant validity reflects the degree to which test scores have the \u2018correct\u2019 patterns of associations with other variables The conceptual foundation of a&hellip; <a class=\"continue\" href=\"https:\/\/www.amyork.ca\/academic\/zz\/psychological-measurement\/convergent-and-discriminant-validity-evidence\/\">Continue Reading<span> Convergent and discriminant validity evidence<\/span><\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[107],"tags":[],"_links":{"self":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts\/4286"}],"collection":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/comments?post=4286"}],"version-history":[{"count":0,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts\/4286\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/media?parent=4286"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/categories?post=4286"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/tags?post=4286"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}