Changes in Japan in the Last Fifteen Years, by Noah Smith

Changes in Japan in the Last Fifteen Years. Noah Smith. Mar 2018, https://twitter.com/Noahpinion

1/I have been coming to Japan since 2002.

Since that time, the country has changed enormously. Year-to-year the changes are small, but looking back, they really add up.

Here are some of the things that have changed.

2/One of the biggest changes is diversity. Especially in Tokyo.

Thanks to the tourism boom, the place is jam-packed with non-Japanese people. But that's not nearly all of it.

3/There are non-Japanese people working everywhere in Tokyo. Latin American chefs in yakitori restaurants. Swedish clerks in clothing stores. Indian staff in electronics stores. Chinese exchange students behind cash registers. Chinese salary workers in company offices.

4/Foreign languages are commonplace.

In 2002 people were impressed that I could speak a few sentences of Japanese. Now, people are just relieved.

5/The second big thing that has changed is gender roles.

In 2002, housewives were still the norm. Now they're increasingly uncommon.

6/In 2002, "office ladies" would flood lunch restaurants while male salarymen stayed at their desks.

Now, the OLs are mostly gone.

7/Furthermore, there is now open discussion of sexism in Japanese society, whereas 15 years ago there was very little.

8/A third big change is where people live.

15 years ago, there were tons of "parasite singles" living with (and living off of) their parents into their 30s.

Now, there are few. A lot of young people have moved out, and a substantial fraction have gotten roommates.

9/In addition to these three big changes, there have been many minor changes.

With the fall in youth idleness, there are fewer bands, fashion kids,  and other young people engaged in such lifestyles.

10/Styles in Tokyo are plainer and more conservative. A few people still dress wild, but "miniskirt and knee boots" has been replaced with "jeans and sneakers" among normal folks.

Osaka is still more flamboyant.

11/Both Tokyo and Osaka are denser and more built-up. Construction continues apace.

12/Startups and entrepreneurship are more common.

There is a tech entrepreneur class here that didn't exist 15 years ago.

13/America's mystique has mostly worn off. Now it's Japan that has the mystique, with boatloads of American tourist kids racing down the street yelling "OMG I'm in Japan!"


14/Half-Japanese kids are starting to become more visible among groups of schoolkids. Mostly half-white but also some half-black. And I'm sure lots of half-Chinese or half-Korean or half-Vietnamese kids I didn't even notice.


15/In general, Japan has changed in ways I'd expect from globalization and economic liberalization.

The one thing that hasn't changed: the prices.

I can still get a Lipton milk tea in any convenience store for 105 yen!

(end)

Women made more negative attributions about, & experienced diminished desire to affiliate with, female targets wearing (vs. not wearing) cosmetics. This penalty was specific to female observers, mediated by decreases in perceived trustworthiness, & driven by less desirable women

DelPriore, D. J., Bradshaw, H. K., & Hill, S. E. (2018). Appearance Enhancement Produces a Strategic Beautification Penalty Among Women. Evolutionary Behavioral Sciences, http://dx.doi.org/10.1037/ebs0000118

Abstract: Previous research demonstrates that women’s beauty is rewarded across a myriad of social contexts, especially by men. Accordingly, from a functional perspective, another woman’s attractiveness can signal competitive disadvantage—and evoke negative responses—among female observers. Further, because the benefits of beauty are rewarded based on superficial qualities rather than on merit or performance, women may perceive same-sex others who use appearance enhancement to gain advantages as being dishonest or manipulative. We examined these possibilities across four experiments testing whether college-aged women impose a strategic beautification penalty (SBP) on female targets that have enhanced their appearances with cosmetics. We found that women made more negative attributions about, and experienced diminished desire to affiliate with, female targets wearing (vs. not wearing) cosmetics. The SBP was: specific to female observers (Experiment 2); mediated by decreases in perceived trustworthiness (Experiment 3); and driven by less desirable women (Experiment 4). Importantly, the negative effects of beautification effort extended beyond the increased physical attractiveness that resulted from this effort. The results suggest that engaging in appearance enhancement can produce unintended negative consequences for relationships between women.


Check also: The Causes and Consequences of Women’s Competitive Beautification. Danielle J. DelPriore, Marjorie L. Prokosch, and Sarah E. Hill. The Oxford Handbook of Women and Competition, edited by Maryanne L. Fisher. https://www.bipartisanalliance.com/2017/08/the-causes-and-consequences-of-womens.html

Local mating markets in humans and non-human animals. Ronald Noë. Behavioral Ecology and Sociobiology, October 2017, 71:148.
https://www.bipartisanalliance.com/2017/09/local-mating-markets-in-humans-and-non.html

The Reversed Gender Gap in Education and Assortative Mating in Europe. De Hauw, Yolien, Grow, Andre, and Van Bavel, Jan. European Journal of Population, https://www.bipartisanalliance.com/2017/09/highly-educated-women-tend-to-partner.html

Marzoli, D., Havlícek, J. and Roberts, S. C. (2017), Human mating strategies: from past causes to present consequences. WIREs Cognitive Science, e1456. doi:10.1002/wcs.1456. https://www.bipartisanalliance.com/2017/09/enhancing-mens-perception-of-their-own.html

Beauty, Effort, and Misrepresentation: How Beauty Work Affects Judgments of Moral Character and Consumer Preferences. Adriana Samper Linyun W Yang Michelle E Daniels. Journal of Consumer Research, ucx116, https://doi.org/10.1093/jcr/ucx116. https://www.bipartisanalliance.com/2017/11/beauty-effort-and-misrepresentation-how.html

Behavioral display of lumbar curvature in response to the opposite sex. Zeynep Şenveli Bilkent University, Graduate Program in Neuroscience - Master's degree thesis. https://www.bipartisanalliance.com/2017/08/behavioral-display-of-lumbar-curvature.html

Macaques show human-like curiosity: Willingness to pay (or to lose) to obtain information that provides no instrumental or strategic benefit, with the amount to pay scaling with the amount of information

Monkeys are Curious about Counterfactual Outcomes. Maya Zhe Wang, Benjamin Hayden. bioRxiv, https://doi.org/10.1101/291708

Abstract: While many non-human animals show basic exploratory behaviors, it remains unclear whether any animals possess human-like curiosity. We propose that human-like curiosity satisfies three formal criteria: (1) willingness to pay (or to sacrifice reward) to obtain information, (2) that the information provides no instrumental or strategic benefit (and the subject understands this), and (3) the amount the subject is willing to pay scales with the amount of information available. Although previous work, including our own, demonstrates that some animals will sacrifice juice rewards for information, that information normally predicts upcoming rewards and their ostensible curiosity may therefore be a byproduct of reinforcement processes. Here we get around this potential confound by showing that macaques sacrifice juice to obtain information about counterfactual outcomes (outcomes that could have occurred had the subject chosen differently). Moreover, willingness-to-pay scales with the information (Shannon entropy) offered by the counterfactual option. These results demonstrate human-like curiosity in non-human animals according to our strict criteria, which circumvent several confounds associated with less stringent criteria.

h/t: https://twitter.com/DegenRolf

We tested the hypothesis that we humans feel disgust when reminded of our animal nature. Some disgusting things may remind us of our animal nature, but they are not disgusting because they do so

Kollareth, D., & Russell, J. A. (2018). Even unpleasant reminders that you are an animal need not disgust you. Emotion, 18(2), 304-312. http://dx.doi.org/10.1037/emo0000365

Abstract: Three studies (Ns = 200, 400, 400) tested the hypothesis that we humans feel disgust when reminded of our animal nature. Participants verbally rated their disgust reaction to pictures of humans engaged in various unpleasant actions. For pictures of events that present danger or suffering, accompanied by an explicit and vivid reminder that animals face the same situation, participants reported fear and sadness rather than disgust. For pictures of events that present a norm violation, an explicit animal reminder (relative to a human picture alone) did not lead to a consequent increment in disgust. For pictures of events that present a physically disgusting contamination, an explicit animal reminder (relative to a human picture alone) led to a decrement in disgust. Thus, not all unpleasant animal reminders are disgusting. Some disgusting things may remind us of our animal nature, but they are not disgusting because they do so.

h/t: https://twitter.com/DegenRolf

Retaliation on a voodoo doll symbolizing an abusive supervisor restores justice

Righting a wrong: Retaliation on a voodoo doll symbolizing an abusive supervisor restores justice. Lindie H.Liang et al. The Leadership Quarterly, https://doi.org/10.1016/j.leaqua.2018.01.004

Abstract: When a subordinate receives abusive treatment from a supervisor, a natural response is to retaliate against the supervisor. Although retaliation is dysfunctional and should be discouraged, we examine the potential functional role retaliation plays in terms of alleviating the negative consequences of abusive supervision on subordinate justice perceptions. Based on the notion that retaliation following mistreatment can restore justice for victims, we propose a model whereby retaliation following abusive supervision alleviates the negative effect of abusive supervision on subordinate justice perceptions. In two experimental studies (Study 1 and 2), whereby we manipulated abusive supervision and subordinate symbolic retaliation—in particular, harming a voodoo doll that represents the abusive supervisor—we found general support for our predictions. Theoretical and practical implications are discussed.

Not using smartphones in the bedroom increases happiness and quality of life; risk of smartphone addiction decreases when smartphones are left outside the bedroom; sleeping without smartphones improves sleep, relationships, focus and wellbeing

Sleeping with the frenemy: How restricting ‘bedroom use’ of smartphones impacts happiness and wellbeing. Nicola Hughes, Jolanta Burke. Computers in Human Behavior, https://doi.org/10.1016/j.chb.2018.03.047

Highlights:
•    Not using smartphones in the bedroom increases happiness and quality of life
•    Risk of smartphone addiction decreases when smartphones are left outside the bedroom
•    Going to bed without smartphones in the room improves quality of sleep
•    Sleeping without smartphones improves sleep, relationships, focus and wellbeing
•    93.6% of participants “might” or “would” consider not sleeping with their phone again

Abstract: Smartphone technology has dramatically changed the way people interact with the physical and online world. Research shows both positive and negative impacts of smartphone and social platform use. Positive outcomes relate to social capital and engagement, while negative impacts result from compulsive usage, negative comparisons and the stress of being ‘always on’. Little evidence is available regarding wellbeing impacts of smartphone use at particular times of day. This study measures the impact of overnight smartphone use on wellbeing. Experimental group participants abstained from smartphone use in the bedroom for one week. The Subjective Happiness Scale, Quality of Life Scale, Smartphone Addiction Scale and Intensity & Time Affect Survey were issued at the beginning and end of the week. Paired sample T-Tests compared pre and post intervention participant surveys scores. It was hypothesised that subjective wellbeing would increase. In three out of four measures (SAS-SV, SHS and QOLS) the hypothesis was upheld, although impacts were relatively small. 93.6% of experimental group participants said they “might’ or “would” consider self-imposing intervention conditions moving forward, suggesting that participants experienced greater benefits not measurable through the questionnaires selected. Some qualitative analysis supports exploration of findings. Further research to explore other wellbeing impacts is encouraged.

Keywords: Smartphone; social media; social overload; subjective wellbeing; cyber psychology; positive psychology


Check also Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction. Samuel P. L. Veissière and Moriah Stendel. Front. Psychol., 20 February 2018 | https://doi.org/10.3389/fpsyg.2018.00141

Abstract: We present a deflationary account of smartphone addiction by situating this purportedly antisocial phenomenon within the fundamentally social dispositions of our species. While we agree with contemporary critics that the hyper-connectedness and unpredictable rewards of mobile technology can modulate negative affect, we propose to place the locus of addiction on an evolutionarily older mechanism: the human need to monitor and be monitored by others. Drawing from key findings in evolutionary anthropology and the cognitive science of religion, we articulate a hypernatural monitoring model of smartphone addiction grounded in a general social rehearsal theory of human cognition. Building on recent predictive-processing views of perception and addiction in cognitive neuroscience, we describe the role of social reward anticipation and prediction errors in mediating dysfunctional smartphone use. We conclude with insights from contemplative philosophies and harm-reduction models on finding the right rituals for honoring social connections and setting intentional protocols for the consumption of social information.

Adolescents Who Experienced Negative Alcohol-Related Consequences Are Willing to Experience These Consequences Again in the Future

Wicki, M., Mallett, K. A., Delgrande Jordan, M., Reavy, R., Turrisi, R., Archimi, A., & Kuntsche, E. (2018). Adolescents Who Experienced Negative Alcohol-Related Consequences Are Willing to Experience These Consequences Again in the Future. Experimental and Clinical Psychopharmacology. http://dx.doi.org/10.1037/pha0000184

Abstract: Alcohol use and risky single occasion drinking are common among adolescents and are associated with a higher risk of various negative social, physical, academic, or sexual consequences. Studies have shown that among college students, willingness to experience negative consequences is associated with a higher likelihood of experiencing these consequences in the future. However, it remains unclear how experiencing negative consequences influences adolescents’ willingness to experience them again. Based on a representative sample of 1,333 alcohol-using 14- to 15-year-olds (47.9% female), a path model was used to examine the associations between risky drinking, negative social and physical consequences, and willingness to experience the specific consequence in the future. As hypothesized, more frequent risky drinking was positively associated with experiencing negative consequences (i.e., saying or doing embarrassing things, regretted sexual experiences, impairment of schoolwork, problems with parents/friends, accident or injury, hangover, vomiting, memory lapses). Contrary to our second hypothesis, adolescents who experienced a negative consequence were also consistently willing to experience it in the future. Findings suggest that adolescents may see the experience of negative consequences as a necessary evil to attain the positive consequences. Prevention efforts may benefit from focusing on ways of attaining positive consequences by promoting alternatives to engaging in risky drinking practices, as well as reducing negative consequences (e.g., by promoting protective behavioral strategies).

Check also Hangover and Risk for Alcohol Use Disorders: Existing Evidence and Potential Mechanisms. Thomas M. Piasecki, Brandon M. Robertson, and Amee J. Epler. Curr Drug Abuse Rev. 2010 Jun; 3(2): 92–102. DOI: 10.2174/1874473711003020092

Abstract: Hangover may be related to propensity to develop alcohol use disorders (AUDs). However, the etiological role, if any, played by hangover in AUD is unclear. From a motivational perspective, hangover can be construed as either a deterrent to future alcohol consumption or a setting event for negative reinforcement that could promote deviant drinking practices (e.g., “hair-of-the-dog” drinking). Hangover could be related to AUD risk even if it does not play a direct role in promoting or inhibiting near-term drinking. For example, measures of hangover might serve as symptoms of AUD or as markers of individual differences that more directly account for AUD risk. Empirical evidence (though usually indirect) exists to support contentions that hangover is related to both risk for and protection from AUD. In this article, we briefly address variation in assessment strategies in existing hangover research because measures of hangover frequency and hangover susceptibility may prove to have different correlates. Next, we review the existing, limited evidence on relations between hangover and AUD risk. Finally, we sketch a variety of theoretically-informed hypotheses that might help delineate productive lines of inquiry for this emerging field.

Keywords: hangover, alcohol, alcohol use disorder, risk, individual differences

Willpower theories predict outcomes across domains of academics, health, goal progress, interpersonal relationships, and well‐being. Generally, limited willpower theorists' belief in their limited capacity results in poorer outcomes, particularly during times of high demand

Lay theories of willpower. Zoë Francis, Veronika Job. Social and Personality Psychology Compass, https://doi.org/10.1111/spc3.12381

Abstract: Some people believe that willpower relies on a limited resource and that performing cognitive work (such as using self‐control) results in mental fatigue. Others believe that willpower is nonlimited and that performing cognitive work instead prepares and energizes them for more. These differing lay theories of willpower determine whether or not one's self‐control performance actually does decrease or increase after use, with only limited willpower theorists showing a decrease (the ego depletion effect). Due to the self‐control requirements of everyday life, willpower theories also predict outcomes across domains of academics, health, goal progress, interpersonal relationships, and well‐being. Generally, limited willpower theorists' belief in their limited capacity results in poorer outcomes, particularly during times of high demand. By understanding how willpower theories form and function, interventions that encourage nonlimited willpower theories may be created to improve people's performance and well‐being.

The Unresponsive Avenger: More Evidence That Disinterested Third Parties Do Not Punish Altruistically

Pedersen, Eric J., William B McAuliffe, and Michael E McCullough 2018. “The Unresponsive Avenger: More Evidence That Disinterested Third Parties Do Not Punish Altruistically”. PsyArXiv. March 28. doi:10.1037/xge0000410

Abstract: Many social scientists believe humans possess an evolved motivation to punish violations of norms—including norm violations that do not harm them directly. However, most empirical evidence for so-called altruistic punishment comes from experimental economics games that create experimental demand for third-party punishment, raising the possibility that the third-party punishment uncovered in these experiments has been motivated by a desire to appear concerned about social norms rather than by actual concern about upholding them. Here we present the results of five experiments in which we used an aggression paradigm to contrast second-party and third-party punishment with minimal experimental demand. We also summarize the results of these experiments meta-analytically. We found robust evidence that subjects who were insulted by a stranger experienced anger and punished the insulter. To a lesser degree, subjects who witnessed a friend receive an insult also became angry and punished the insulter. In contrast, we found robust evidence that subjects who witnessed a stranger receive an insult did not punish the insulter, although they did experience modest amounts of anger. In only one experiment did we find any punishment on behalf of a stranger, and this result could plausibly be explained by the desire to escape the moral censure of other bystanders. Our results suggest that experimental designs that rely on demand-laden methods to test hypotheses about third-party punishment may have overstated the case for the existence of this trait.

New Zealand Psychologists Board - Guidelines on the use of Psychometric Tests - February 2013, updated March 2015 --

1
Guidelines on the use of Psychometric Tests
February 2013, updated March 2015
The purpose of the Board’s “best-practice” guidelines
Practice guidelines recommend specific professional conduct for psychologists to educate and
inform practice. Guidelines are recommendations rather than mandatory standards, but
supplement the Code of Ethics which is the highest and most aspirational regulatory document.
The Code of Ethics (the Code) delineates the manner in which psychologists ought to carry out
their practice. All other statements of how psychologists should or must conduct their practice
must be consistent with this document and its ethical principles of respect for the dignity of
persons, responsible caring, integrity in relationships and responsibility to society. Guidelines
adopted by the Psychologists Board (the Board) support psychologists in providing competent
and ethical practice by translating or expanding on the Code in relation to more specific aspects
of their professional behaviour.
By integrating the principles of the Code and current specialised knowledge in an area of
practice, the Board develops guidelines to support quality services for the benefit of consumers
and to protect the public. It is incumbent upon psychologists to be familiar with any Board
guidelines relevant to each area in which they practise. Guidelines are not definitive, binding, or
enforceable by themselves. They have the least authority of any of the regulatory documents.
However, a disciplinary body may use the guidelines in evaluating a psychologist’s knowledge
and competency. Guidelines that are relevant to a particular area in which a psychologist has
chosen to practise help to define competent and skilled professional behaviour. Practice that is
inconsistent with relevant guidelines may represent unskilled practice.
Objectives and limitations of the “Psychometric” Guidelines
These guidelines offer guidance and best practice suggestions to support psychologists to uphold
high ethical and professional standards when using psychometric tests. The guidelines do not
attempt to inform the application of practice to each of the many specialised fields where
psychometrics are used, nor does it attempt to research the literature pertaining to each of these
specialisms. The guidelines have been informed by a distillation of ethical principles integrated
with consultation submissions from experts throughout the psychology community in New Zealand.
The submissions during the multiple consultations to guide the development of these guidelines
indicate that some aspects of practice are controversial or that there is a wide range of opinion.
These guidelines do not attempt to prescribe what should or should not occur but focus more on
the generic ethical and professional use. The guidelines assume psychological training and
expertise and are not intended to inform or train practitioners from other disciplines who would like
to or do use psychometrics. Psychologists cannot directly control people from other disciplines
who may also use psychometrics, but can offer leadership and expertise which may influence
others towards good practice.
2
Contents
Executive Summary .......................................................................................................................... 3
What is meant by “a psychometric test”? .................................................................................. 5
Safe and ethical use of psychometric assessment procedures ............................................. 6
To uphold the highest standards of accuracy and fairness when administering tests ...... 7
Informed consent ............................................................................................................................. 9
Interpretation .................................................................................................................................... 9
The use of symptom validity testing, reliability and dissimilation ........................................... 10
Third party observers ...................................................................................................................... 13
Cultural considerations.................................................................................................................. 14
Responsible reporting of psychometric results ......................................................................... 15
The potential risks in the use of tests and how those risks can be mitigated ...................... 16
Keeping psychometric records ................................................................................................... 17
Responsible use and the protection of the intellectual property of tests ........................... 18
Training in the use of psychometrics ........................................................................................... 19
Use of psychometrics by students during training ................................................................... 20
Special issues related to infants and children .......................................................................... 20
Special issues related to the assessment of adults with intellectual
disability………………………………………………………………………………………………….23
Use of psychometric assessment for purposes where the psychologist is an expert witness 23
Purchase of tests ............................................................................................................................. 25
Computer based assessment and Internet communications of test results ...................... 25
Use of Psychometrics in research ................................................................................................ 26
References ....................................................................................................................................... 27
3
Executive Summary
Psychometric tests are structured and standardised assessment procedures which enable a
psychologist to measure aspects of a client’s functioning. The standardised administration allows
comparison of that client with others who have completed the same assessment procedure. If the
test has been developed on a similar population as the client then the test outcome may be
considered a valid measure of that attribute. Psychometric tools vary in the degree of formality
between structured observations, such as questionnaires, to prescribed tasks which are
administered under carefully controlled standardised conditions. Predominantly these guidelines
address the use of standardised tests by psychologists as applied to their practice with individuals.
The Executive Summary of the key principles presents the generic standards which may be
expected to be upheld should it be relevant to a complaint or competence concern. The
indexed sections that follow in the remainder of the document will allow a more detailed
consideration for those interested in that particular topic. These more detailed sections often
represent a range of opinions or issues. It is also acknowledged that the content may be very
basic to those who are experienced but has been included because it has either been raised as
an issue or a complaint or competence concern has illustrated that it cannot be taken for
granted. These issues are raised as matters to take into account but are not intended to be
prescriptive or define exactly how a practitioner should proceed. This will always be determined
by professional judgement in the context of the formulation of the issues.
Principles of use
The use of psychometrics should ordinarily comply with the following principles. However there
may be circumstances where deviation is justified.
Principle 1: The use of the information
Information derived from psychometric testing is usually collated with information from other
sources to inform the assessment of a client, rather than being considered an adequate
assessment in its own right. Triangulated information which is consistent may strengthen
confidence in the test findings but conversely discrepant information may necessitate
interpretation or further enquiry. Care should be taken to not place undue weight on the
psychometric findings or to predict performance in another setting based on the test outcome
unless there is evidence that test has predictive validity for performance in that other situation of
interest.
Principle 2: Competence to use that test
Psychologists should ensure that they are competent and trained to use a test prior to using it with
a client. The degree of training required will vary depending on the sophistication of the test but
should enable the integrity of the standardised administration to be maintained and a wellinformed
interpretation of results made.
Principle 3: Informed consent
Informed consent should be gained from the client prior to the test being administered. The
consent should define who has the right to receive this information. This consent should be in
writing, particularly where the information is to be given to a third party.
Principle 4: Justification for the test selection
The psychologist should be able to justify the selection of tests. This may mean refining the referral
question. Tests selected should be fit for the purpose required.
Principle 5: Interpretation and the limitations of test information
The psychologist has an ethical obligation to ensure that information arising from psychometric
testing is not misused. Therefore there must be careful attention to interpretation and defining the
limitations of that information.
4
Principle 6: Avoiding over-testing
Testing should not be any more frequent or intrusive than necessary to gain the required
information. Excessive exposure of psychometric tools to a client or allowing the client to become
overly familiar with tests should be avoided to minimise practice effects and to preserve the utility
of that test should it be needed to assess that client in the future.
Principle 7: Follow administration instructions
The psychologist should use the prescribed and standardised procedures to administer a test.
Measuring functioning in this controlled way enables comparison of that client’s performance
with others who have been tested in the same way. Deviations from those conditions should be
stated as constraints on the interpretation of the results.
Principle 8: Third party observer
Standardised administration of tests means a psychologist should not allow a third party observer
to be present when administering tests. However two sets of circumstances may override this
standard.
The Health and Disability Commissioner’s Code of Patient Rights grants a client the right to have a
support person present, should they request it. A psychologist should attempt to explain why that
is not optimal in a psychometric testing session (it may reduce the validity of the test results and
may threaten the test integrity). If the client insists, the psychologist may choose to allow this and
work to reduce the disruption as much as possible or the psychologist may choose to decline
proceeding with the assessment.
There are other circumstances where the presence of a support person is necessary to enable the
testing to occur, such as the need for an interpreter or where there is a need for emotional or
physical assistance.
In judgements on allowing or declining the presence of a third party observer, a client centred
approach should be taken and then any perceived constraint to interpretation stated when
reporting the results.
Principle 9: Cultural safety
Psychometric tools used with New Zealand populations commonly use normative samples from
other populations. Test results should be interpreted with this limitation stated if applicable as
populations from different cultures may have systematically different test profiles.
Principle 10: Release of psychometric data
Psychologists should release the interpreted results only to those who have a legitimate right to
receive that information. Interpreted results take in to account any constraints on the validity of
the test results and have been integrated with collateral information and observations. Raw data
should not be accessible by those who are not trained to interpret it and by those who do not
have consent to have access. A client who requests the release of their test records may elect a
psychologist who can receive the raw data on their behalf and who could assist in any
interpretation.
Principle 11: Reporting results
Reported results should be fit for the purpose and targeted at the objectives of the assessment. In
both written and oral feedback, the limitations of the assessment should be communicated and
the results presented in a form that is understandable to the audience.
5
Principle 12: Decision making criteria
The more serious the consequences arising from an assessment, the more stringent any decisionmaking
criteria need to be. Differential hypotheses or diagnoses should be considered, avoiding
bias or foregone conclusions towards predicted outcomes.
Principle 13: Assessing for third party
When assessing under contract to a third party, the psychologist has an independent expert role,
and therefore should take care to not be influenced by the contracting organisation to produce
or confirm a preconceived outcome. Conversely the psychologist is not an advocate for the
client. The consent process should make explicit to whom the report belongs and with whom the
information will be shared. The consequences of choosing to not participate in the assessment as
well as the possible outcomes of the test results may also need to be discussed.
Principle 14: Confidential storage of test data
Psychometric test data should be stored in a secure and confidential manner. The test data
should not be accessible to those who are not trained to interpret and should be viewed only by
those who have consent from the client to access. In accordance with the Health (Retention of
Information) Code, psychometric data, including raw data must be kept for at least ten years
from the latest time that a client was seen.
Principle 15: Security of psychometric tools
Psychometric tests should be protected from unauthorised access to preserve copyright restraints
and to avoid misuse of the tests (to the extent that the psychologist has control). The contents of
tests should be safe-guarded. It is acknowledged that the proliferation of information about
psychometrics in the public domain, particularly through the internet may compromise this
intention.
Principle 16: Use of psychometrics by those in training
Psychometric tools should only be used by those in training under close supervision. The consent
gaining process should make transparent the status of the trainee and that the supervising
psychologist is accountable. The accountable psychologist has a duty to ensure that standards
are not compromised.
Timelines for review
It is recognised that the use of psychometrics is constantly evolving. These guidelines will be
regularly reviewed. Therefore any aspect which is considered outmoded or to inaccurately
represent current “best practice” should be drawn to the Board’s attention and these guidelines
can be updated. Once finalised it is intended that the guidelines will be reviewed routinely at two
year intervals.
What is meant by “a psychometric test”?
A psychometric test is a structured and standardised measurement of cognitive, behavioural or
emotional functioning including (but not restricted to) performance tasks, structured behaviour
samples, self-report inventories or checklists, test record forms, or other materials used in the
evaluation of an individual or a group of individuals. It is normally designed to be administered
under carefully controlled or standardised conditions that include systematic scoring protocols.
The psychometric test provides a measure of performance which allows inferences about the
individual to be drawn based on that sample of behaviour, as it allows comparison with a larger
population. Psychometric tests may also allow classification or the placement of individuals within
a range of possible measures.
Psychometrics are used in a wide range of settings to assist psychologists to understand and
predict behaviour, then to use this information to make decisions and guide future action. For
example, in an occupational setting the psychometrics may assist selection for employment
placement or the trajectory of development within an organisation. Educational evaluations and
resource allocation may be informed by testing. Within a clinical setting, psychometrics may be
used for a wide range of purposes including diagnosis, clinical decision making and prognosis.
6
Neuropsychologists offer expert assessment using psychometrics to diagnose and assess
functionality where a client is hypothesised to have a brain dysfunction or neurological disorder.
In most situations the psychologist would use tests to supplement other information gathered,
rather than using psychometrics as the only source of information. By using a standardised test,
the psychologist is able to add structured information to the informal information gained from
other sources such as interview and observation. Psychometric questionnaires may be used as a
systematic and efficient way of gathering information and/ or screening to assist triage and to
identify areas for in-depth investigation. Carefully selected psychometric tools may hone in on
aspects of functioning to identify aptitudes and abilities, to inform diagnosis or to predict
performance on correlated real-life tasks.
Safe and ethical use of psychometric assessment procedures
Psychologists must only use test instrument that they are competent to administer and interpret,
unless working under the close supervision of professionals with appropriate training and
experience. The qualifications specified by the test’s producers or in the test manual must be
complied with. The training required to enable competent administration of a test will vary
depending on the complexity of the test.
The psychologist should gain informed consent prior to undertaking the assessment and only
report the results to those whom the client has given permission to inform and who have a right to
know. This is discussed more fully in a later section.
Consistent with normal professional practice, psychologists are accountable for the contents of
any report that they sign. Psychologists must select appropriate assessment instruments and
procedures for the objectives of the assessment and should be able to justify the selection and
interpretation of tests if required.
Psychometric testing should occur in a structured and controlled environment. It is not
appropriate to give the client the test to take away to complete elsewhere. Such uncontrolled
administration would open up possibilities of the client taking advice from others how to complete
the test and reduce the test security. Furthermore, the testing process may elicit reactions in some
clients which are best observed and addressed.
Psychometric tests vary in the rigour with which they have been developed. A psychometric test
should be valid (the degree to which evidence and theory support the interpretations and
relevance of test score in the proposed use of the test)), reliable (that is, the consistency of
measurements obtained on a test when the testing procedure is repeated on a population of
individuals or groups) and sensitive (able to differentiate with regard to the attributes of interest)1.
Test measurement presupposes that individuals exhibit some degree of stability in their behaviour
and the attributes of interest. However repeated measurements will show some variability which
can be referred to as measurement error around a hypothetical “true” score. Errors of
measurement will consist of random and unpredictable errors as well as potential systematic
errors, for example a client with high test-anxiety may systematically underperform which
introduces a construct-irrelevant variance. A distinction can be made between measurement
error which arises within a client and those arising from external factors, such as changes on the
administration or unintended distraction. In a repeated assessment situation a change in score
from one occasion to another may not be considered an error of measurement but may be
hypothesised to arise from an intervention or healing or other process. In this situation the reliability
of this observed measure is of interest. Whatever the test situation, the confidence that can be
placed on the test outcome should be commented on as it is essential to an interpretation of a
test result2. Reliability and validity are not independent constructs as high error measurement will
inevitably reduce the validity of the resultant measures.
A more rigorous test is one that has been shown to be valid with reliability when tested against a
larger sample and preferably a wider normative population that could reasonably be considered
1 Refer to Eatwell, J. and Wilson, I. (2007) for a full discussion of the psychometric qualities of tests.
2 AERA, APA, National Council on Measurement in Education (1999)
7
as representative of the population from which the test-taker is drawn. A constraint on the
interpretation of tests used on New Zealand populations is that there are few tests which have
been developed and validated against New Zealand populations. Therefore the normative data
which the New Zealand client is being compared with may be based on a limited sample of the
population from the country in which the test was developed.
The type of measure3 used determines the comparisons that the test enables. The type of
questionnaire, will determine the nature of the information generated, ranging from structured
interview data to classification into categories. Percentile scores or ratings allow an ordering that
indicates how that individual has scored compared to the comparison group, and are generally
easily understood by laypeople. However a cautionary note is that percentiles should not be
considered equal units of measurement as the ordering will tend to exaggerate differences near
the mean and may collapse differences at the extreme. Standard scores or interval scale systems
of reporting results will show the individual’s performance relative to the normal distribution. Raw
scores cannot be compared directly with those of others whereas standard scores have been
transformed to reflect variations with respect to a specific group on a specific test, in terms of
standard deviation from the mean..
The results of assessments may have a substantial impact on the client. Therefore it is important
that psychologists uphold the highest standards of accuracy and fairness when administering
psychological test instruments. The ethical responsibility extends to striving to ensure that others
both understand the limitations of assessments and that they do not misuse the results. The
psychologist should take care to consider the various factors which may have changed the
outcome scores, such as cultural or age factors, practice effects, or contextual factors.
In a formal report used for legal or decision making purposes, the assessing psychologist should
state their training and experience with this type of assessment to allow the reader to judge with
what authority the psychologist speaks.
To uphold the highest standards of accuracy and fairness when
administering tests
A key question pertaining to the selection of a test is “What is the purpose of the assessment?” The
psychologist should not necessarily accept the referral question at face value but may be
advised to discuss the referral question with the referrer to clarify the objectives of the assessment
before planning the assessment. The referral question may not be appropriate or answerable in
the form it is initially presented. The psychologist should resist pressure to provide statements on
causation or draw inferences beyond the evidence. The psychometric assessment only provides a
snapshot of the functioning of the client at the time of the assessment and therefore does not
substitute for a formulation about that client based on much broader and triangulated
information from multiple sources.
Decisions about testing should be based on a thorough analysis of the client’s requirements and
the purpose that the assessment is addressing. For example, in an employment selection process
the test should have validity for measures that correlate well with the occupational competencies
of interest and be pitched at an appropriate level of difficulty to differentiate between individuals
on the target attributes of interest. In an educational setting the purpose of the assessment may
be to measure performance relative to the age cohort or the target skills required for successful
learning. A psychological assessment often proceeds by the psychologist formulating hypotheses
about the client which are then tested by gathering data to support or disprove. Psychometrics
used for this purpose should be carefully selected to inform the constructs of interest. The
advantages and disadvantages of using any particular test within a proposed assessment
strategy should be carefully considered so that there is a reasoned justification for the use of any
one test.
Care should be taken when reporting the results of tests when the client is intellectually disabled.
For example reporting age equivalent scores for adults with developmental delay may be
3O’Connor, F. (1993)
8
misleading unless carefully interpreted as the person has life experiences and biological
development matching their chronological age.
The psychometric assessment will be of most value if the client is motivated to do their best, is
interested and engaged, and the relationship with the assessing psychologist enables openness.
This will be promoted by the client understanding the purpose of the assessment and perceiving
the assessor as professional in their conduct. A professional approach in this context would require
the assessing psychologist to be respectful, friendly and seek to put the client at ease but also to
be perceived as neutral and unbiased. The psychologist should not be overly familiar or negative.
If a client is observed to be anxious, distrustful or unmotivated, this should be noted as a constraint
on performance which can then be taken into account in the interpretation.
An individual may be given multiple tests at one time. Optimally these tests should be carefully
selected to answer specific clinical questions pertaining to that client. Administering a standard
set of tests, often referred to as a “battery” of tests, increases the risk of “over-testing” the client,
thus reducing the usefulness of any future assessment of that client. The psychologist will be
guided in their selection by the specific hypotheses or lines of enquiry for that client and the
purpose of the assessment, for example whether the assessment is directed at information
gathering, triage, predicting future performance in a role, comparison with peers, cognitive
screening, a comprehensive psychological assessment to build up a profile of abilities and
weaknesses, or some other purpose.
If the assessment is for the purpose of neuropsychological diagnostics, the assessor may have a
good knowledge (prior to starting the assessment) of the likely or possible neuropsychological
profile for that client and the norms for the selected tests. These advance hypotheses may enable
the assessor to know how to respond or redirect the assessment as information comes to hand.
Ideally an assessment proceeds using the least intrusive way to gather information available,
keeping the psychometric testing to an optimal level (which may mean keeping it to a minimum).
The value of having working hypotheses has to be balanced with the need to keep an open
mind to avoid bias and predetermination.
Any test selected should be fit for the purpose for which it is intended, both with regard to the use
for which it was designed and validated, and for the client group on which it was developed. The
statistical properties of that test apply only to the intended use, administered in the prescribed
manner, and scored according to instructions.
If the norms for a specific test are to be used, the directions and instructions specified in the
manual must be strictly followed to enable the results to be compared to the results of others. If
there is reason to vary the administration of the test, this should be fully recorded with the
rationale for this variation. It is recognised that under some circumstances tests may be altered in
how they are administered because of the clinical situation. The potential impact of the variation
on the scores should be noted and allowance made in the interpretation of results. The
psychologist should also be alert to possible constraints on interpretation, such as where a client
may vary from the test’s normative group (e.g., by ethnicity, age, disability, gender, or other
attributes).
If the client has been assessed previously, the psychologist should be alert to the possibility that
practice effects may influence the client’s performance. These risks may be managed
proactively, such as by using an alternative test with similar properties, using an older version of
the test or by statistically allowing for this.
Sources of unintended distraction should be removed prior to starting the assessment session,
including turning off cell phones and pagers.
Psychologists should ideally use the most current version of a test, unless there is a particular
clinical reason for using an earlier version. Clients should be welcomed and briefed in a manner
that alleviates anxiety. If the client continues to show anxiety, appropriate assistance may need
to given. This may include providing encouragement and emotional reassurance, taking a tea
break, or suggesting that a support person be consulted or invited to the assessment. The
inclusion of a third party into the assessment situation is discussed more fully below.
9
Informed consent
Psychologists must gain informed consent from a client before proceeding with a psychometric
assessment. In accordance with the Health Information Privacy Code and the Privacy Act, this
means informing the client of the purpose of the assessment, how the information may be used
and who may have access to those results. Extra care may need to be taken with those with
language barriers or intellectual disability to ensure that the client understands what is involved.
Any constraints to confidentiality should be explained at this time. If the report is being prepared
for a third party under contract, it may not be possible to state who will have access and for what
purpose the report may be used other than in general terms.
Gaining informed consent may include discussing the consequences of taking, versus not taking
the test, so that an informed choice can be made by the client. For an intellectually disabled
client this may mean talking through possible consequences of being found incompetent. By
informing the client of potential risks and benefits arising from the assessment, the client is able to
exercise their right to choose cognisant of likely consequences. The psychologist should be alert
to any limitation of a client’s capacity to give or decline consent, including medical or mental
diagnoses, physical or cognitive incapacity, language or cultural barriers.
In a situation where the client is unable to consent, the psychologist may need to consult with
relevant family, the person’s legal counsel, the enduring power of attorney, or seek a court order.
This may be particularly relevant to assessments determining functional competence or where the
client is disabled.
Where children are involved, obtaining informed consent is likely to require the consent of parents
or guardian. When the subject of the assessment is a child of separated parents, informed
consent should ordinarily be sought from both parents, but if this is not possible, from the legal
guardian or the parent who has custody. The age that a child becomes their own decision maker
to give informed consent varies with the development of that child depending on the child’s
ability to understand the purpose.
In accordance with the Privacy Act and the related Health information Privacy Code,
psychologists seek to collect only that information which is germane to the purpose for which
informed consent has been obtained. The consent is likely to be specific for the time period and
the situation in which the assessment occurs. If circumstances change, if there is a significant
lapse in time, if additional parties request a copy of the report, or if the report is to be used for a
different purpose than initially consented, further written informed consent should be sought.
The parties who may legitimately receive the test results should be clearly identified. The
psychologist’s ethical obligation to ensure the client gives informed consent for the release of
information about them extends to situations where there is a request for a report or psychometric
results arising from an historical assessment. Generally only interpreted results should be released,
once the appropriate consent has been obtained and should not be communicated to anybody
else without the prior informed consent of the client.
Interpretation
Psychologists should interpret test scores in conjunction with other collateral information, such as
information from interview, information given by relatives and friends, observational data and
previous assessment reports. Far reaching conclusions or diagnoses which have high impact on
the client’s life should not be made on the basis of a psychometric test or a one-off assessment.
Reasonable alternative explanations for the results should be considered.
Constraints on the assessment that may affect the interpretation of results should be stated. There
are multiple factors that can compromise a client’s performance on a psychometric test
including factors such as the client’s mental state or physical state, side effects of medication,
language or cultural barriers, educational limitations to understanding, the testing environment,
the interaction with the tester, fatigue, and the client’s recent and historic background. The
psychologist may need to differentiate between impairment of functioning which is transient and
short term, or that which is more enduring. Constraints on interpretation may also include the lack
10
of normative data for a client’s ethnic, cultural or social group. Where constraints have been
identified, these should be recorded and allowed for in the interpretation of results.
If the same psychometric instrument has been undertaken by the client previously, the assessing
psychologist should be mindful of any practice effects and allow for that in the interpretation of
results. Comparison of scores on repeat assessments may give useful information.
Computer-interpreted test results or a computer generated report is not adequate alone as an
assessment report. The psychologist has a responsibility to evaluate the computer based
interpretation of test performance or the computer-generated report in the light of additional
information, clinical observations, and other known data about the particular client.
The psychologist should report derived scores, such as standard scores, percentiles and ageequivalents
with care to mitigate any risk arising from use or misuse by readers who lack
understanding or training in the use of psychometrics. For example, these measures may be
regarded as more fixed and enduring than what is appropriate. Cautions about the limitations on
the reliability and validity of such scores should be explained.
The psychologist should avoid over-generalising the results of one test to traits or characteristics
not measured by the test. The extrapolation of that result to predict success in real life situations,
such as academic success, current and future employment status, performance of daily living
tasks, medication management and ability to drive should only be done if it has been established
that test result is predictive of those daily functions, or that performance on that test is highly
correlated with performance in some other setting.
The use of symptom validity testing, reliability and dissimilation
The interpretation of psychometric assessment results relies on the results being valid for that
individual. The value of an assessment to meet the purpose for which it has been completed
depends on the quality of the test data on which it draws, including the client’s motivation to
adhere to the test requirements. Effort or motivation indicates the client is performing at their
capacity, demonstrating a willingness to comply with explicit or implicit instructions with regard to
speed, accuracy or other performance requirement. It is not uni-dimensional, but is a concept
which may be assessed and inferred from observations of behaviour. Optimally a client
demonstrates his or her best effort in a testing situation. Effort can vary from poor to outstanding
as part of natural variation. In gaining informed consent, it should be explained to the client that
they should perform to the best of their ability and that tests of effort may be included in the
assessment if applicable.
In any assessment in which there are known advantages or potential advantages to a client
presenting him or herself in a particular way, then the psychologist should consider and comment
on this issue directly. There may also be unexplained discrepancies between client self-report,
various sources of collateral information, observed behaviour and changes in functionality over
time. In these circumstances the psychologist may choose to include tests which are sensitive to
detecting the effort applied by the client.
Symptom validity testing, also known as effort testing, is intended to give reliable and valid indices
which are sensitive to distortions in motivation. A4 distinction can be made between effort tests in
a cognitive assessment context, which is assessing performance validity (such tests clarify the
extent to which a client’s test performance is or is not an accurate reflection of their actual level
of ability) whereas the term symptom validity more specifically refers to the accuracy of selfreported
symptoms as, for example, elicited by tests which ask the client to report attributes and
may include embedded validity scales. A failure on a performance validity test means that the
client has performed poorly below a suitable cut-off or below their capability as determined by
other criteria.
4 Larrabee, 2012.
11
Opinion on the routine inclusion of such tests is divided. Some psychologists advocate that it is
“best practice” to include effort tests routinely as part of a clinical assessment of cognitive
function, particularly where clients are involved in litigation or claiming financial benefits for
disability, and that failure to assess this would need to be justified. In this context it is argued that
effort testing both indicates the validity of the assessment and also may protect the client from
unfair criticism. Decisions with regard to the allocation of treatment, rehabilitation, financial
support or medication or culpability in a legal context may be based on the test results. Research
has shown that external incentives can be influential on performance.
If psychometric tests are used to specifically assess effort during a cognitive assessment, these
tests should be dispersed throughout an assessment. The following ethical considerations5 apply:
• Only well researched effort tests should be used. The selection may need to be guided by
client disability.
• Multiple effort tests should be used to reduce the likelihood of false positive diagnosis of
poor effort. Research6 has suggested that using more than one performance validity tests
reduces the incidence of false positive errors.
• As with all psychometric assessments, performance validity testing must be undertaken
with careful attention to administering tests in standard ways and noting any constraints
on interpretation.
• The psychologist should explain to the client that it is important to provide their best effort
and to report symptoms and problems accurately, as failure to do so can often be
detected.
• If testing occurs over an extended session or over more than one day, the psychologist
should be alert to the possibility that the motivation to succeed may fluctuate. If effort
tests are used, they should be distributed throughout the testing sessions.
• The psychologist should also examine performance patterns to ensure they make
biological and psychometric sense.
• The clinical inference of depressed effort should be made on a convergence of evidence
rather than simply one or more effort tests alone.
• Information from various sources should be integrated and compared for consistency,
including behavioural observations, interview data, collateral records, collateral interviews,
and psychological and neuropsychological test results.
• An effort test only shows behaviourally there was poor performance on that test but this
may occur for various reasons. The assessing psychologist should consider all reasonable
possible differential diagnoses or explanations for the observed behaviour, and list the
evidence for each of these alternative explanations.
• Conclusions should be stated explicitly and clearly. Psychologists have an ethical
responsibility to report assessment results fairly, accurately and objectively. Referring to the
client’s assessed inconsistent motivation in critical or pejorative terms is to be avoided as
to do so would be likely to breach the Code of Ethics.
The assessment of performance validity requires considering the pattern of performance across
multiple measures. One approach which has been widely accepted gives an operational
approach to diagnosing that poor effort has been applied.7 Specified criteria are that there is an
external incentive clearly identified, a negative response bias, and that behaviours cannot be
accounted for by psychiatric, neurological or developmental factors. The evidence for the
negative response bias would be drawn from below chance performance on forced choice
measures of cognitive functioning; performance on one or more well-validated psychometric
tests designed to measure feigning; discrepancy between test data and known patterns of brain
functioning; discrepancy between test data and observed behaviour; discrepancy between test
data and reliable collateral data; and discrepancy between test data and documented
background history. There may be implausible changes in test scores across repeated
examinations and unusual or bizarre errors observed during the interview.8 The self-report data is
5 Iverson, 2006, 2007.
6 Cited by Larrabee (2012)
7 Slick, D. et al (1999)
8 British Psychological Society, 2009.
12
also considered for discrepancies with other information gathered and for indications of
exaggerated psychological dysfunction.
If a psychometric test indicates that less than optimal effort has been applied or has been
inconsistently applied, the first question is “why?” This question may not be answerable. There are
various factors9 which may contribute to a client demonstrating poor effort in an assessment
situation, i.e. poor performance validity, including dementia states with fluctuating attention span,
sensory or motor impairment, abnormal arousal states, severe psychiatric disorder, poor
communication or understanding of the demands of testing, and physical factors such as
musculoskeletal injuries (e.g. peripheral neuropathy). However effort tests are designed to be very
easy and performance is not easily affected by other factors. Clients with diagnosed neurological
conditions often still show performance levels in the valid range of performance validity tests. Test
performance10 has been shown to be not affected by pain, fatigue, psychiatric disorders or
depression and anxiety states.
A conservative interpretation is that low effort on one part of the test series may indicate all other
results are under-representing the person’s abilities. It also means that the test data should not be
relied on to give a valid indication of performance and therefore may not be interpreted in a
meaningful way.
When self-reports are not consistent with other data, an assessing psychologist should not assume
deliberate intention to mislead, i.e. poor symptom validity. The client may inform the psychologist
in good faith and have no intention to deceive but may have become highly focussed on their
difficulties, may (falsely) attribute pre-existing symptoms to an accident, report a higher than
actual pre-morbid level of functioning, catastrophise current symptoms or have difficulty reporting
current functioning accurately.
Distinction can be made between the following terms. However these states may be overlapping
in a client’s presentation:
• Symptom validity: the accuracy or truthfulness of the client’s behavioural presentation and
self-reported symptoms.
• Performance validity: the accuracy or truthfulness of the client’s performance on tests
(usually neuropsychological measures).
• Response bias: an attempt to mislead the examiner through inaccurate or incomplete
responses or effort.
• Malingering: the intentional production of false or exaggerated symptoms, motivated by
external incentives.
• Dissimulation: the intentional misrepresentation or falsification of symptoms by over
representation or under representation of a true set of symptoms in an attempt to appear
dissimilar from one’s true state.
• Factitious disorder: physical or psychological symptoms that are intentionally produced or
feigned in order to assume the sick role. In a factitious disorder, the symptoms are
motivated by internal emotional and psychological issues, which lead the person to
maintain a sick role, rather than the client being motivated by external incentives.
• Somatisation disorder: recurring, multiple, clinically significant somatic complaints which
cannot be fully explained by any known general medical condition or the direct effects of
a substance.
9 British Psychological Society, 2009.
10 Research cited by Larrabee, 2012.
13
• Conversion disorder: symptoms or deficits affecting voluntary motor or sensory function
suggestive of a neurological or other general medical condition which is considered to be
triggered by internal conflicts or emotional states.
• Pain disorder: diagnosed when pain is the predominant focus of the clinical presentation
and is of sufficient severity to cause significant distress or impairment in functioning. The
pain is considered to be caused or maintained by psychological factors.
In assessments carried out for clinical and rehabilitation purposes, there is an obligation for the
psychologists to provide feedback to the client even when there is lack of effort detected on
cognitive measures and/or an exaggeration of emotional and behavioural symptoms is
hypothesised. This discussion may lead to clearer identification of the reasons for
underperformance or exaggeration of symptoms which can then be targeted in
therapy/rehabilitation. For example, feedback on a perceived lack of effort may be structured
around a discussion in the form of “what factors can get in the way of you performing at your
best?”
Third party observers
The interpretation of psychometric results is based on the test being administered in a
standardised way to allow comparison with the normative population who have also all
undertaken the test in the same standardised manner. These standardised conditions are likely to
be compromised by the presence of a third party observer (TPO). As a general principle,
wherever possible the standard conditions should be preserved. However the Health and
Disability Commissioner’s Code of Patient Rights gives a client the legal right to request a support
person to be present and there may be client presentations where testing is enabled by the
presence of a TPO.
In accordance with Rule 8 of the Code of Health and Disability Services’ Consumers’ Rights, a
health consumer may request a support person to be present. Although there are some
exceptions to this right (specifically, if safety may be compromised, another consumer’s rights
may be unreasonably infringed or if declining the request for a support person is reasonable in the
circumstances), this rule has been interpreted by the Office of the Health and Disability
Commissioner as meaning that the subject of an assessment does have the right to expect a
support person to be present during psychometric testing. As the Code of Rights has legal status
which overrides the Board’s Code of Ethics, this rule supersedes the ethical obligations of the
psychologist to avoid having a TPO present, should such a challenge be made to what is
deemed best practice for a psychologist assessor. If a client is requesting that a support person
remains present during psychometric assessment, the psychologist assessor should attempt to
engage the client in a collaborative relationship and explain the disadvantages of having an
observer-support person present. However if the client is not persuaded, their right to have a
support person prevails. The psychologist may decide to decline to conduct the assessment
rather than compromise their practice.
The greatest validity is to be obtained when the client is motivated to cooperate with the assessor
to perform in an optimal manner in compliance with the instructions in a standardised controlled
environment. The presence of a TPO risks the validity of the test results by potentially impacting on
the client’s motivation, altering the rapport with the examiner, and may distort the response to
test items by the distraction both from the physical presence and the internal processes
stimulated by the awareness of the TPO’s presence.
The effects of an observer being present on the client’s performance is likely to vary depending
on the nature of the assessment, the purpose for which it is be done, the manner of observation,
and the client’s sensitivity to being observed. However research studies have tended to
consistently show lower performance with a TPO. Furthermore the impact is variable and
unpredictable so cannot be controlled or allowed for in the interpretation beyond placing less
reliance on the results. The psychologist should refer to the research literature pertaining to
individual tests to gain an understanding of the evidence on the relative sensitivity to a TPO being
present. Individuals will be more or less sensitive to these observed effects. Tests measuring
14
attention, sustained concentration, verbal fluency, learning and memory have all been shown to
be sensitive to the impact of having an observer present. Some clients are less likely to share
personal information if they consider others are observing so that interviews may produce less
information in that circumstance.
There is also a risk to test security, which is against the ethical obligation for psychologists to make
all reasonable efforts to maintain the integrity and confidentiality of test materials. The
psychologist has no control over how a TPO may use the observations of tests allowing the
possibility of test misuse, including misinterpreting poor performance or coaching.
The request to allow a third party observer may arise for a variety of reasons, including (but not
limited to) the desire to have a support person or whanau present, a child wanting a parent or
caregiver to be present, a trainee wanting to observe the psychologist, or there may be a desire
to record a session as part of gaining evidence for legal purposes. There may be circumstances
where the presence of a TPO is deemed necessary to make the assessment situation accessible.
In these situations a client centred approach should be taken. For example, there may be barriers
arising from the lack of linguistic knowledge or expression, a physical inability to see or hear, or a
lack of the emotional security needed to engage in the cognitive processes. Situations which may
be enhanced by the presence of another include (but are not limited to) a parent with an
anxious child, an intellectually disabled client requiring support, a migrant from a substantially
different culture, a sign or language interpreter assisting a client to overcome those barriers, or an
assistant who is able to facilitate physical accessibility.
In the many situations where the benefits of having a TPO present outweigh the prohibitions about
doing so, then the psychologist should document their reasons and rationale, including the
client’s consent and preferences. Ways to minimise the impact of observation on the validity and
fairness should be carefully considered. Possible steps include seating the observer behind the
client, and ensuring the observer consents to not speaking or otherwise influencing the client
during the assessment. If the TPO needs to be more actively involved, their participation should
facilitate but not undermine or impair the assessment. The psychologist should warn the client that
the TPO may affect the results when obtaining consent and document this as a possible limitation
on interpretation. Where there is variation from the standard conditions, this should be
documented and allowed for in the interpretation of the results.
As a profession it behoves all psychologists to continue to educate non-psychologists the reasons
why psychometric testing should be conducted privately and confidentially without observation
to protect the usefulness of the tests and the inherent intellectual property.
Cultural considerations
Optimally a psychometric test is used that has been developed and shows validity for measuring
the attribute of interest in a population that fairly represents the client. Relevant comparison
groups provide a normal distribution with which the individual’s score can be compared. It is
essential that the normative group used is appropriate to the context and purpose for which the
test is being used, to avoid misleading conclusions. The comparison group needs to be as similar
as possible to the situation in which the client’s behaviour is being predicted or measured against.
It should be noted that few psychometric instruments have New Zealand normative standards
available. One group of researchers who developed New Zealand normative data for the Rey
Complex Figure Test found a number of significant differences between their results and the
American standardisation data, but no overall ethnic differences.11
There is a paucity of research on the validity of psychometric instruments used with various cultural
groups but three research studies illustrate the ways interpretation could be misleading.
Research12 has shown that for example young Māori men with no known history of traumatic brain
injury may show as much as five scaled score points difference between subtests on the Wechsler
Adult Intelligence Scale-Revised, with relatively lowered Vocabulary scores but with Block Design
results elevated by as much as one standard deviation when compared with others of that age.
11 Fernando, K.,Chard, L., Butcher,M., and McKay, C.(2003)
12 Ogden, J. and McFarlane-Nathan, 1997 cited in Ogden, J., 2007.
15
Secondly, administrators of the SF-36 health survey could assume that all cultures interpret the
health questions the same way but research13 has shown that Pacific and older Maori
conceptualise their health differently. Research14 on rehabilitation outcomes after traumatic brain
injury which used cognitive assessments before and after intervention concluded that Māori,
Pacific and Pakeha groups all benefitted from the programme but that the years of education
and English as a second language were confounding factors in interpreting the psychometric
data. There were other cultural differences between the groups in their psychological outcomes.
These three studies are illustrative of the principle that psychologists should be alert to the cultural
bias of tests if they have been developed and normed for a different cultural population.
Meaningful differences in performance can be found between the average performance of
different ethnic groups or between men and women, raising the possibility of adverse outcomes
arising if the test results are used for decision making. These examples suggest that differences
between cultural groups may not be intuitively obvious.
Differences in performance may arise from factors including, but not limited to, differences in
socio-economic conditions impacting on educational opportunities, where the language of the
psychometric tool is different from the native language of the test taker, or discomfort/ perceived
threat in the test situation. There may be differences among cultures in familiarity with language
used or images used in test items. Simply translating the test into the client’s native language may
not render the test valid as there may not be cross-cultural construct equivalence.
Where it is not possible to use psychometrics with normative data matched to the client and it is
not possible to provide a psychologist who is culturally matched with the person to be assessed,
attention should be given15 to make the setting comfortable for the client. Cultural advice may
be sought on how to put the client at ease. Tests should be chosen with care and performance
interpreted tentatively, using collaborative information from a wide range of other sources to
assess pre-morbid and current abilities, including the observations of others, as well as work and
education records. Tests which rely heavily on formal western education and include culturally
alien concepts should (optimally) be avoided where assessing Māori or Pacific peoples until the
cultural bias pertaining to various tests is clarified.
A self-reported ethnicity of Māori cannot be assumed to mean the same thing to all individuals in
that category. Some psychometric development has focussed on measuring Māori knowledge 16
and dimensions of Māori identity and cultural engagement.17
If translators are employed, the accuracy of the translation may be problematic as it introduces
variance both in delivering the instructions and in recording the response. Wherever possible an
assessment should be conducted by an assessor speaking the same language as the person
being assessed.
Responsible reporting of psychometric results
Generally only interpreted results should be released to those whom the client has given consent
to receive the information. Where a third party has contracted the assessment, it may not be
possible to identify the recipients of the report except in general terms. Furthermore, some
contracted reports prohibit the direct release of the report and its findings, such as those
contracted by the Family Court which strictly controls who has access to the report arising from a
contracted assessment. Complex or potentially litigious reports should be reviewed in supervision
prior to release as a safeguard for the psychologist and the client. Psychologists who are in
training or at an early stage of their practice may also be advised to review all assessments
routinely.
13 Scott, K. et al, 2000.
14 Faleafa, M., 2009.
15 Ogden, J. 2007.
16 Thomas, D. 1988.
17 Houkamau, C. And Sibley, C.2010.
16
The discussion of the results should be directed to what the interested readers need to know. A
concise, targeted assessment directed at the objectives of the assessment is more likely to be
helpful than an overly inclusive, long, poorly directed review of functioning. Interpreted results
would ordinarily explain the test outcome integrated with other data, any limitations or constraints
pertaining to the outcome, and also give either the confidence with which the results can be
considered (sometimes expressed as a confidence interval) or likely reliability of the
measurements. The test outcome is likely to compare this client with others on the attribute or
measure of interest.
If the assessment has been conducted in an organisational or employment setting, the purpose
of the assessment and to whom the results will be made available should be established in the
consent gathering phase, and then the focus and range of reporting of results previously agreed
and consented should be carefully adhered to. The raw data should not be left with the
organisation. The assessing psychologist should attempt to anticipate any potential misuse of the
test and use appropriate risk mitigation strategies such as stating the limits of the use of the test,
whom to contact should fuller information and interpretation be required, and to assure the client
and the contracting organisation that the original test material has been stored securely.
In a health setting, the psychometric assessment should add value to the treatment and
management planning. The report should give a good summary of the functioning as measured
by the assessment and whatever recommendations arise from that assessment to improve the
rehabilitation or quality of life of the person. If the report is within an educational setting, caution
should be exercised. A report may stay on a child’s record for a long time and be influential in
decisions about that child, such as the allocation of resources. A child’s performance relative to
peers may change considerably as development occurs. In both health and educational settings,
any diagnosis or labelling of a client should be made with caution as this potentially stigmatising
action can have wide reaching impacts and potentially may be destructive. The reporting of
results for an intellectually disabled client needs to avoid misleading age comparisons as the
chronological age is likely to not match developmental profiles.
Oral feedback to a client should be presented in a constructive and supportive manner, using
language which is understandable for the audience. The results of a psychometric assessment
should acknowledge the limitations and constraints to interpretation. The reporting should make
clear what is factual information and what is professional opinion or interpretive comment. The
technical and linguistic levels of written reports should be appropriate for the level of
understanding of the recipients. The weighting of the test result as compared to other information
should be explained. Written reports should also include a summary and any recommendations
arising. If the report is being used to inform a decision, such as recruitment selection, any
limitations to the predictive validity should be explained.
Psychologists have an ethical obligation to strive to ensure assessment results are understood and
used only for the purpose for which consent was obtained. This may become difficult to uphold if
the information is given to a third party organisation or contractor. As much as possible the
constraints on the control over the use of the information should be explained as part of obtaining
informed consent at the beginning of the assessment.
The potential risks in the use of tests and how those risks can be mitigated
When completing an assessment, a psychologist reports on sensitive information in a manner that
may have profound and long lasting effects on the person who was assessed. It behoves the
psychologist to strive to conduct the assessment as ethically and competently as circumstances
allow. The more serious a decision that may arise from an assessment, the more stringent the
decision making criteria need to be. If a long lasting decision is to be decided on the basis of the
assessment, then a high standard of validation of the results should apply.
Care should be taken to consider differential diagnoses and lines of enquiry when interpreting
results of assessments. Mental health factors such as depression, anxiety and thought disorder
may contribute to current functional impairment. Wherever possible information should be sought
from multiple sources including self-report, behavioural observation, rating scales, clinical
17
interview, interviews and reports from collateral sources of information (such as family, friends,
employer, other clinicians). The assessment data should optimally inform of the duration of the
presenting problems or abilities as compared to pre-morbid functioning.
Confirmatory bias is a phenomenon observed when psychologists differentially seek and assign
weight to supportive evidence at the expense of plausible, alternative explanations for the
obtained test results. For example, a psychologist may attribute working memory, psychomotor
and executive deficits to a historic mild traumatic brain injury while ignoring a pre-accident history
of learning problems and past and on-going problems with substance abuse. Alternately an initial
impression may lead the psychologist to have a low threshold for information that supports initial
working hypothesis, while ignoring, discounting or minimising data that leads to a different
interpretation. All possible hypotheses need to be entertained and examined.
Psychologists also need to be aware of the scatter of tests scores in normal healthy children and
adults, especially the prevalence of low scores. This will help psychologists to avoid relying
excessively on isolated low test scores when formulating professional opinions.
The psychologist may be biased by a desire to be an advocate or to supply the answer
requested by the contractor. There is a risk of the psychologist being “captured” by the
contracting organisation, either through the contractor setting prior expectations as to the
expected outcomes or by predetermining the nature and content of the assessment (rather than
the psychologist exercising his or her own professional judgement). Conversely the client may
place strong pressure, either explicitly or covertly, on the psychologist to present a particular set of
findings. The psychologist should protect their professional independence and integrity by
preserving the right to draw up a formulation based on the assessment results. Similarly while the
contractor is entitled to spell out the questions to be addressed in the assessment, the
psychologist should retain the right to choose the most professionally appropriate way of
proceeding. The psychologist may need to make transparent to the client the independence of
his or her professional opinion.
A psychometric assessment may be misleading if there is insufficient attention to the constraints
on interpretation. Normally an assessment is a multistep process which includes integrating and
comparing information from the background referral data, an interview with the subject of the
assessment, behavioural observations , and collateral information gained by interviewing
significant others which may include reports on functioning in other settings. Discrepant or
inconsistent information may need to be investigated further or may prompt repeat assessment
after a period of time.
Repeat assessments may cause the subject of the assessment to be overly familiar with the tasks
required in the assessment. Some tests are more susceptible to practice effects which could be
predicted to boost performance.
Relationship with the assessor may act as a constraint or limitation on the person’s ability to
comply with an assessment.
Keeping psychometric records
Psychometric test results, consisting of both the raw data and the interpreted results, should be
safeguarded to preserve confidentiality and to avoid those who are not trained to use them in a
manner that could be misleading. Raw data arising from psychometric assessments should be
retained in a secure file. In an organisational or health service setting this is likely to require that
the psychologist keep psychometric assessment records in separate, secure filing systems rather
than including them in the client’s main personal or health records (which may be accessed by
other professionals and non-professionals). In some settings it may be appropriate to keep the
psychometric records on the main file in a sealed and labelled envelope.
In accordance with the Health (Retention of Health Information) Regulations 1996, all client
records must be retained for ten years minimum from the date of the latest client contact. Client
records include the raw data from psychometric assessments.
18
Test results used for research purposes should not identify the test subject by name. Names and
personal identifiers should be removed from databases of results that have been archived for
research purposes or where the data has been used for the development of norms.
Should a client undertake repeated psychometric assessment, it is highly desirable that the
psychologist assessor has access to the results of the previous assessments for comparison.
Psychometric records could be released to the psychologist concerned with the client’s
permission but raw data should not be released directly to the client or any other untrained
person.
Use of Online Services
There is a trend towards the use of online services to deliver psychometric testing. This may involve
the administration, scoring, interpretation, and/or storage of a psychometric tool or of test data
using cloud-based storage facilities. Such services raise a potential issue with regard to the
interpretation of the results, which should always be integrated with other information about the
functional behaviour of the test subject.
Secondly there are risks about the security of the information arising which should be considered.
In this situation the psychologist remains accountable for the secure storage of client information
and to ensure the information remains retrievable (for a minimum period of ten years, or longer if
the results remain relevant). The psychologist should be aware that there are risks associated with
using off-site storage (covered in the Board’s Best Practice Guidelines on Record Keeping).
Responsible use and the protection of the intellectual property of tests
Psychologists should protect the security of standardised and controlled tests where possible. This
includes respecting copyright restraints and preventing unauthorised access to psychometric
instruments. It is acknowledged that the ability to control the distribution and access to
standardised tests has been considerably compromised by the wide dissemination of information
about tests on the internet. Professional reference material, conference presentations,
advertisements promoting the use of specific tests by distribution companies, YouTube clips and
training videos are sources of information that a motivated client could access to pre-empt an
assessment.
Test users who are not trained or lack the requisite skills may use tests inappropriately. Where
necessary, psychologists may need to educate colleagues and the public about the appropriate
use of tests and the need to safeguard the confidentiality of the contents of tests. In a
multidisciplinary team context, other professional colleagues who are not trained in
psychometrics may request the right to access or use the tests. While the use of non-standardised
checklists or systematic observations may be used by non-psychologist colleagues, any tests
relying on standardised administration and interpretation which have been validated against
normative groups should not be made available. To do so would risk degrading the integrity of
the test and the possible generation of misleading information. A psychologist in a team context
should take a lead role in safeguarding psychometric instruments and ensuring that any tests are
used ethically and only by those who have had appropriate training.
Ordinarily, audio and video recording should not occur when psychometric measures are being
administered in order to protect the integrity of the tests measures (see exceptions noted in the
next paragraph). Audio and video recording allows ready access to the test materials by the
general population and violates the protection of test materials. This can provide people with an
undue advantage when tests are administered at another time, for example for recruitment
purposes, and can also allow effort measures to be easily identified by people who may later use
this information in a manner unintended by the test developers. For example, if the content of
performance validity measures became widely known, then they may become invalid indicators
of effort. Clients may request a recording to be made of an assessment, for example as a memory
aid or to refer to later should there be a dispute over the outcome of an assessment. This is
particularly likely in a legal context. The psychologist should respectfully explain why this is not
allowable in order to protect the integrity of the test by avoiding the contents being common
19
knowledge and thus ensuring its continued usefulness. There is also an ethical obligation to
protect the intellectual property of the psychometric tools. The psychologist should be alert to the
use of Smartphone and other technology which may enable a client to record a test situation in a
covert manner.
There may be circumstances where sound ethical reasons for recording may be established, such
as for teaching and training purposes, for research, for quality control purposes and as part of the
development of psychometric instruments. In such circumstances, ethical practice would be
safeguarded by attention to consent processes, safeguarding of the recording, documentation
of the rationale for the action taken, plus transparent consideration of the risks and any risk
mitigation.
In a situation where assessment data needs to be made available, such as in a legal dispute, a
complaint investigation or a competence review, then the test materials should only be made
available to another psychologist. If the psychologist is put under legal pressure to release raw
data to a non-psychologist, the practitioner can decline the request. This may lead the applicant
requester to ask a judge to evaluate the relative worth of the arguments for considering the
psychometric data as privileged, and therefore not admissible, versus the common law right for
people to know the evidence against which they may be judged. The objections to the release of
the raw data could be presented to the court, such as the risk of misinterpretation by an
untrained person, protecting the integrity of the test and the intellectual property rights of the test
distributors. If the judge orders that the raw data is to be released, the psychologist must oblige as
to refuse to do so would be in contempt of the court. However where possible the perceived risks
arising from the release should be mitigated as much as possible. If a psychologist is unsure of his
or her legal right to withhold psychometric data, it may be advisable to obtain independent legal
advice. Often such advice is available through an indemnity insurer.
Psychometric assessments used in a legal dispute may also lead to a situation where a second
expert assessor is employed to critique the first assessment. The original assessor may be requested
to release the raw data to that second assessor. Any such release should only occur with the
client’s permission. The critiquing psychologist should be mindful of the risks of interpreting data
out of context.
Test techniques should not be described publicly as that may impair their usefulness.
Training in the use of psychometrics
Psychologists should be both knowledgeable and experienced in the use of a specific
psychometric test before employing it with clients. Specialist training for a particular test may be
required and where appropriate the psychologist should also do refresher training.
Training for the use of psychometrics should include (but is not restricted to) knowledge of:
• Basic psychometric principles, procedures and the technical requirements of tests
(including reliability, validity and standardisation).
• Specific tests and the purpose to which they may be used to enable the proper
interpretation of test results.
• Relevant theories and models of ability, personality and other psychological constructs, or
of psychopathology to both inform the choice of tests and the interpretation of results.
• The range of tests and test suppliers relevant to the specialist domain of practice. There is
on-going development of new tests on the market. An organisation such as NZCER which
is not aligned to any particular test distributor and is non-profit making can offer
independent advice about the range of tests available for a particular purpose. NZCER
also offers a lending library through which a registered user can access a manual to study
prior to purchasing a particular test.
• Skills for specific assessment procedures and instruments, including the standardised
conditions pertaining to the administration of a particular test.
• Ethical and legal issues about the use of tests, the reporting of the results and the secure
storage of test data.
• Professional responsibilities for the proper use and storage of test materials.
20
• The qualifications and experience specified by the test suppliers for each psychometric
tool. NZCER categorises tests (levels A, B, Csp, C and D) depending on the level of prerequisite
training and psychometric knowledge which is deemed necessary to enable the
psychologist to use the test competently and ethically.
Use of psychometrics by students during training
Students who are being trained in the use of psychometrics must be closely supervised by an
experienced and qualified assessor. Students should not be able to access or purchase tests
directly but may do so under the supervision of the responsible psychologist who remains
accountable for any assessment and for the test security.
The obtaining of informed consent should include explicit information about the training status of
the assessor and also should name the psychologist who is accountable. The responsibility of the
supervisor or overseeing psychologist extends to ensuring that all stages of the assessment are not
unduly compromised or reduced in quality by the student undertaking the administration.
Particular care should be taken in situations involving subjects who may pose additional
challenges, such as the assessment of children and with the verbal feedback to the client and
their family. The report on the assessment should be countersigned by the supervising
psychologist.
The student should be well informed on the theory and statistical properties underpinning the test
prior to working directly with the psychometric test. Practice with the administration of the test
should only be undertaken in controlled clinical situations. For example, it is not appropriate for
the student to practise on a family member but it may be possible for students to practise on
each other in a classroom laboratory situation.
Special issues related to infants and children
Any assessment of infants and children should take into account developmental factors. There is
also a range of normal variation of the rate of cognitive development at any age level.
The assessment of children should be based on multiple sources of information, including
behavioural observations and collateral data. Psychometric tests with infants consist of structured
observations and guided interviews of the caregivers. Although there is some psychometric
testing using standardised and normative reference tests with young children, the evidence for
the use of these tests is relatively weak, particularly where the child has a disability. Concerns18
raised about the use of standardised norm-referenced tests with young children include the
following reservations:
• They have low treatment validity as they do not directly inform intervention.
• They are not universally designed or adaptable, for example for use with children with
sensory challenges.
• It is difficult to capture the real life behaviours/skills of young children. Contrived activities
with unfamiliar people are not an effective way to indicate functional competencies.
• The emphasis on scripted standardised procedures to preserve reliability and validity of
normative measurement is incompatible with the typical behaviour of young children who
are likely to seek to explore their environment and have limited interest in staying still or
attending to adult controlled tasks.
• They do little to facilitate collaboration with parents or educators.
• Children with disabilities are often excluded from group data and therefore the norms may
not apply.
If the purpose of the assessment is to develop an intervention plan, other assessment methods
such as direct observation in natural settings and structured interviews should be used.
18 Macy and Bagnato, 2010.
21
The Ministry of Health19 operational guideline for the assessment of children with intellectual
disability notes that “it is not always possible or useful to psychometrically assess children under the
age of 6” (page 9). Psychologists must have a thorough understanding of the limitations of the use
of psychometrics with infants and children who are developmentally delayed or with disabilities.
Infants and children should always be supported to allow them to show their best performance.
Infants and young children are co-regulated by their caregivers and attachment figures, rather
than by strangers such as an assessor.
When assessing a child a wide range of other factors may impact on performance and should be
commented on. Internal factors that should be considered include illness, effects of medications,
nutritional states, hunger, sleeping habits, physical mobility, motivational interest level, anxiety,
stress, and the ability to self-regulate internal and external worlds. External factors may also be
very influential, including the duration of a test taking session, heat, cold, noise, the time of day,
family expectations and maternal mental health.
Some language modification or subtest selection to suit the child is sometimes required.
Testing needs to be fit for purpose and presented in appropriate time intervals. For example if the
child is 6 years and under, or those who have attention difficulties are best seen in 1 hour sessions
before midday. It may be optimal to split the session time down into even shorter blocks of time. If
a child is unwell, it may be advisable to delay the assessment until back to normal health.
A psychologist assessor is attempting to measure development while it is occurring. A ‘one-off’
psychometric assessment may not provide an accurate sample of psychological status, as for
example, it could change the next day when a new skill emerges. Collateral information from
parents and caregivers or direct observation in natural settings may help inform the assessor
whether this presentation is representative or typical.
Overly simplistic interpretation of score results may at best not provide helpful or developmentally
accurate information, and at worst may be potentially damaging and significantly inaccurate.
Written clinical reports may form part of an on-going record in the child’s medical or educational
file and have an enduring impact on the child’s future.
Psychologists should be very careful in their interpretation of test results and only make clinical
judgements within their direct knowledge and experience. A range of other ways to gather
assessment information such as direct observation and structured interviews should be considered
rather than generalising from one-off psychometric assessments about conditions such as
language, dyspraxia, learning disability, Autism Spectrum Disorder, preterm birth and Down
syndrome. It is also important to realise that a particular observed behaviour may have a number
of different explanations.
Extreme care should be taken when agreeing to psychometric assessment for infants and young
children with developmental delay or disabilities:
• What is the purpose?
• What is to be gained?
• Who will manage the initial and possible on-going distress of the caregiver following the
discussion of results?
• Will the assessment results be used to inform the intervention plan?
• Do those to whom the results will be reported fully understand the limitations of such an
assessment?
Use of students to carry out psychometric assessments with infants and young children, whether
for clinical assessment purposes or within research studies needs to be very closely regulated. They
must be properly trained and supervised and have sufficient experience to ensure they can elicit
19 Ministry of Health “Operational Guideline for the Assessment of Intellectual Disability to Access Disability Support
Services” page 9
22
the best performance from the child and communicate appropriately with parents regarding the
child’s performance.
While obtaining informed consent from parents or caregivers remains crucial for children of all
ages, gaining consent and cooperation from older children and teenagers is very important.
Special issues related to assessing adults with an Intellectual disability
Informed consent should also be obtained when assessing an adult (defined as a person over the
age of 18) diagnosed with intellectual disability. The psychologist needs to ensure the client is as
aware as possible regarding the purpose and process of the assessment, and potential
implications of the results. This may mean adapting and presenting information in a more
accessible manner, for example using simpler wording, pictures and/or objects. The psychologists
also need to assess the client’s understanding by asking the client to relate back their perception
of the situation in their own words.
If the client is unable to provide informed consent, even with more accessible information
provided, then consent needs to be sought from a person legally allowed to provide this on their
behalf; this would generally be an appointed Welfare Guardian. Where an adult does not have a
Welfare Guardian, as is commonly the case, a best interest meeting should take place, where
relevant people are consulted and decisions with regards to the client’s best interests are agreed.
In some situations, clients may be placed under a Court Order requirement to engage in and
complete an assessment, and therefore no formal consent from the client is required.
Assessors should note that many psychometric tests did not include adults with intellectual
disabilities amongst the population they based their norms on. This reduces the validity of these
tests for this population and therefore results need to be interpreted with this in mind.
Non-verbal clients can be assessed using language free tests, which have often been developed
for people whose culture / first language may influence their scoring on other assessments.
It may be tempting for an assessor to adapt the test material to make it less complex for their
client by enlarging stimuli or simplifying test material. However any changes are likely to affect the
test validity. For example enlarged test materials would require greater physical movements to
complete the task and thereby affect the speed the person can complete the task and therefore
unfairly impact on the processing speed score. Assessors should not copy or enlarge stimuli or
simplify test material in any way as it would change the presentation or process of the assessment
and therefore make the interpretation of the test difficult. Equally it is unethical to substitute core
subtests based purely on what the assessor believes the client can or cannot do; substitutions
should only be made if subtests are rendered invalid during the assessment (for example, if there
was a significant distraction) as allowed within the test manual directions.
When the assessment is to ascertain whether a person has an intellectual disability or not, it is
essential to complete an IQ assessment (or evidence attempting to do so if the person is unable
to complete one) and an assessment of adaptive daily living skills, as well as obtain evidence that
the person experienced developmental delay in their childhood. It is also important to consider
and assess the potential influence of any contextual factors that may have a negative impact on
a person’s current abilities and subsequent assessment results (such as current mental health
problems or head injuries).
The client may require more time than average to complete an assessment and it can be helpful
to plan to complete longer assessments over a couple of sessions (as close together as possible).
In most circumstances it is best practice to complete an effort test as part of any
neuropsychological assessment. The assessor should ensure the effort test used is suitable for a
person with an intellectual disability, i.e. has a low enough baseline to accommodate low scoring
due to low cognitive functioning rather this outcome being attributed to mood or malingering.
Clients are often used to being acquiescent in their daily lives and this should be kept in mind
when completing assessments. For example, It can be helpful to agree a way for the client to
23
indicate they need a break, such as by raise their hand or tapping the desk, before starting the
assessment to help empower those who are too shy to ask directly.
Individuals with an intellectual disability may benefit from having a support person present during
their assessment and have the right to do so as long as it does not pose any risk to the client, other
people or the assessment process. The presence of a support person may reduce the client’s
anxiety and therefore increase the likelihood that they perform to their best level. As with any third
party observer, it should be stressed prior to the assessment that the support person should not
interfere with the testing in any way and is best positioned seated out of the client’s direct vision.
Some assessment instructions may need to be simplified to allow clients to understand what is
required of them. All clients should first be given the standard instructions and then have the
opportunity to report whether they understand them or not. Any required changes need to be in
keeping with the original intent of the instructions and not provide any further information or clues
regarding accurate task completion. Similarly it would not be appropriate to allow a client more
than the standard number of practice trials on an assessment task. It is important to note when
and how instructions were changed as this information provides useful insight into client abilities.
Any deviation from the standardised instructions may compromise the validity of the test and
should be recorded, as per usual professional practice.
An open-minded approach to neuropsychological diagnostic assessments may be helpful so as
not to bias interpretation of results. If the assessor already holds particular hypotheses then this
may influence the chosen assessments and the way the test behaviour and data are understood.
Further it is worth recognising that an existing diagnosis of intellectual disabilities may over-shadow
other cognitive issues such as dementia and specific brain injury.
The reporting of age equivalent scores is not always appropriate for adults with an intellectual
disability as they can give an inaccurate impression of them fitting neatly into a childlike
developmental stage. The client has life experiences and the biological development of their
chronological age which are beyond the experiences of a typical child sharing their cognitive
abilities.
As in other domains of psychometric application, assessments should only be completed by those
professionals with appropriate training, experience and qualifications to do so. Because of the
particular issues with this special needs group, psychometric results should only be interpreted by
those professionals with the appropriate training, experience and qualifications to do so. Results
should be reported in a clear manner, in their standardised form. Raw data is not reported but is
kept with the original assessment paperwork.
Clients are entitled to feedback on their assessment results and may appreciate a simplified
version of their assessment report where appropriate. With permission from the client, it is often
helpful to feedback results to a nominated person(s) within the client’s support network for
example parent or key-worker.
A resource book has been issued by the Ministry of Health pertaining to the assessment of those
with intellectual disability which is a timely adjunct to the Board’s guidelines. See “Operational
Guideline for the Assessment of Intellectual Disability to Access Disability Support Services
Contracted for People with Intellectual Disability within New Zealand”, published by the Ministry of
Health in 2012.
Use of psychometric assessment for purposes where the psychologist is an
expert witness
A psychologist contracted as an expert assessor may use psychometric assessment as an integral
component of the information gathering. The roles of Family Court assessor, forensic examinations,
cases involving litigation, employment disputes, assessments for disability support and ACC
contracted assessments to review benefit entitlement are some of the roles which can be viewed
as conforming more to a legal paradigm than that of delivering a health service.
24
The roles of therapist and expert assessor are likely to be conflicted. Therefore a treating
psychologist should avoid agreeing to be an expert witness or to perform an evaluation for legal
purposes although may be requested to give evidence of the observed facts or a clinical opinion
of their client, with that person’s consent. A psychologist who is contracted to do an evaluation as
part of a legal or quasi-legal process is answerable to the court or the lawyer or the contracting
agency who engaged his or her services. However this should not compromise the psychologist’s
professional integrity or independence. The psychologist should be careful to resist any explicit or
implicit pressure to influence professional judgement with regard to the process of the assessment
or the outcomes from either the contractor or the subject of the assessment. The psychologist may
not be able to keep the roles separate where there are few with the requisite skills and the clients
may have special needs, e.g. when the client is an intellectually disabled person.
For the psychologist therapist, confidentiality may only be waived by the client or by a court
order, and normally is subject to being privileged information. Privilege in this context means that
the confidentiality of information obtained as part of a therapeutic engagement is protected in
accordance with the Evidence Amendment Act, No 2 1980 (sections 32(3) to 33(4)). Legal
privilege for children and young persons is protected by section 77 of the Children, Young Persons
and their Families Act (1989). By contrast the expert evaluator reports to the lawyer or contractor,
as he or she is acting as their agent. There is a duty to inform the subject of a legal evaluation of
the constraints to confidentiality and the intended use of the product of the assessment.
Whereas the therapist is a care provider, the expert assessor must be neutral, objective and
detached. Whereas the therapist is not so concerned about historical truth as the perception of
the client, the expert evaluator may need to offer an opinion on the validity of the psychological
aspects of the client’s claims. This usually means verifying the client’s reports against other
information sources about the events in question by seeking collateral information, including from
psychometric assessment.
Whereas a therapist–client relationship is based on the principles of beneficence and nonmaleficence,
the expert evaluator strives to gather information and to present objective
information that allows a legal decision maker to reach a just solution to a legal conflict or
determination of entitlement. This may be detrimental to the legal position of the subject of the
examination. The therapist develops a therapeutic alliance with the client, and avoids actions
which may disturb that relationship, while the expert assessor’s role is to assess and report the
findings to a third party who will use that information in an adversarial setting or one which may
be subject to dispute.
If completing an assessment as an expert assessor, the psychologist may need to check with the
contractor or instructing lawyer whether there are any restraints to giving the client feedback.
While it is ordinarily desirable to give feedback, there may be prohibitions against this in some
circumstances.
When providing expert advice to a court or decision making authority, the psychologist should
take care to not exaggerate the attributes of a psychometric test and psychologists should not
go beyond their competencies.
In giving testimony, a psychologist must stay mindful of the confidential nature of the
psychometric tools used and to avoid releasing information about the nature of the test into the
public domain. Protecting misuse of tests includes safeguarding the confidentiality of the test
material, avoiding release of such data and materials to unqualified persons and releasing data
without adequate interpretation. If the psychologist is pressured to supply information about the
tests, he or she should advise the court of the risks of compliance, that is, the potential loss of utility
of that assessment tool.
Where there are clear incentives for the client to be found to have a disability or condition, the
possibility of malingering may need to be evaluated. The DSM IV defines malingering as “the
intentional production of false or grossly exaggerated physical or psychological symptoms,
motivated by external incentives such as avoiding military duty, avoiding work, obtaining financial
25
compensation, evading criminal prosecution or obtaining drugs.20” Assessments for the purpose of
ACC or income protection insurers determining entitlement to benefits are examples where there
could an incentive for the client to exaggerate symptoms or feign disability. International best
practice states that symptom and performance validity testing should be done routinely to
demonstrate this aspect has been actively considered21.
Purchase of tests
Tests can be purchased from various sources, including NZCER, the specific test publisher, or other
internet-based providers. It is recommended that tests be purchased in accordance to the
competency levels set by some test producers and that specific training be undertaken for using
the specific test in a competent and clinically useful way.
Psychologists who wish to purchase tests through NZCER are required to register with that
organisation. This will involve declaring their scope of practice, relevant training and prior
experience. The psychologist will be assigned a level of tests that they are deemed eligible to
access. In this way the access to more complex tests is controlled as a safeguard against the
potential misuse of the tests. Most psychologists are likely to be able to access Level C and D tests,
but this is not an automatic right.
It is a commercial reality that many psychometric instruments (particularly those used in human
development and employment selection) are now available for purchase through the internet in
an uncontrolled way. This may degrade the usefulness of some tests. While psychologists are
bound by their professional obligations to maintain ethical conduct, others may not be so
responsible. It behoves psychologists to lead by example and to demonstrate there is added
value in having a disciplined, scientifically sound approach to an assessment which integrates
(often complex) information to enable useful interpretations and application to the issue of
interest. There may also be a need to educate stakeholders of the potential risks that may arise
from the misuse of psychometrics.
Computer based assessment and Internet communications of test results
Computer based assessments include a range of scenarios, such as “high stake” scenarios where
important decisions rest on the outcome of the assessment (such as recruitment selection),
through to “low stake” scenarios, such as test-takers satisfying their curiosity how they perform on
a measure but where there are no foreseeable consequences of taking the test and there may
be minimal interaction with the psychologist. The test situation may range from being supervised
and password controlled, to being unsupervised where the identity of the test-taker may not be
authenticated.
Informed consent should include clarification of the limitations of internet assessment and if
appropriate, the extent of the relationship with the psychologist who is administering the test. Back
up phone and email contact information may be appropriate if a client requires further
explanation of the intended purpose of the assessment and the potential outcomes of the
assessment.
The psychologist should assess the appropriateness of the use of the internet based test as
compared to a test delivered by an alternative method. The content of the test, the technical
adequacy, and the validity of the test for the desired purpose should all be considered. Particular
care should be used if the norms and psychometric technical data are based on pen and paper
or face to face delivery.
Limitations to interpretations may include the use of uncontrolled, and therefore less standardised,
conditions. It may also be impossible to ensure the true identity of the test taker. There may be less
access to other observational information that could be used as collateral evidence to the test
result.
20 APA (1994), pg 683.
21 British Psychological Society (2009), National Academy of Neuropsychology (2005), American Academy of Clinical
Neuropsychology (2009)
26
Test interpretations should be given in a comprehensible and meaningful form which is fit for the
intended purpose and recipient audience. Due to the difficulty of knowing the impact of
negative feedback, the lack of knowledge of the state of mind of the test-taker, and the difficulty
of providing immediate support if there is a negative reaction to feedback, it may be appropriate
for feedback to include direction on how to obtain support.
The minimum hardware and software to support the test delivery and also the browser necessary
to deliver a test over the internet must be specified. The client must have the appropriate level of
skill and comfort in using the technology for the results to be valid. The computer based test
should not require knowledge, skills or abilities that are irrelevant to the attribute being measured
as these other qualities could act as a barrier to performance on the test.
The user of an online test should have the technical understanding to set up the test and provide
clear information so that the test taker is able to log in and out of the test. Clients should have
access to on-screen help while completing the test.
Use of Psychometrics in research
Research use of psychometrics will also include the development of any new or emerging
psychometric tools.
Researchers using psychometric tests should maintain the same standard of ethical practice as
psychologists working in other specialist areas. This should include abiding by copyright restrictions,
such as not photocopying protocols to avoid purchasing psychometric materials, and reserving
the right of use to those with appropriate psychometric training.
Informed consent should include communicating the purpose of the research, how the individual
respondent’s data will be used and stored, whether or not there is provision for being given
individual feedback or a report on the outcome of the research, and who is responsible for the
research.
Names or other personal identifiers should be removed from archived, stored research data.
27
References
Accident Compensation Corporation (2012). ACC Guidelines for the Use of Psychometric Testing.
Wellington: ACC
American Academy of Clinical Neuropsychology (2001). Policy statement on the presence of
third party observers in neuropsychological assessments. The Clinical Neuropsychologist, vol 15, no
4, 433-439.
American Educational Research (1999, 6th edition) Standards for educational and psychological
testing. Published by American Educational Research Association, American Psychological
Association, and National Council on Measurement in Education. Washington, DC: American
Educational Research Association.
Aronoff, G. et al (2007) Evaluating Malingering in Contested Injury or Illness. Pain Practice, vol 7,
issue 2, 178-204.
APA Committee on Psychological Tests and Assessment (2007). Statement on Third Party
Observers in Psychological testing and Assessment: A framework for decision-making.
Board of Directors (2007). American Academy of Clinical Neuropsychology (AACN) Practice
Guidelines for Neuropsychological Assessment and Consultation. The Clinical Neuropsychologist,
vol 21, no 2, 209-231.
Binder, L. and Rohling, M. (1996). Money matters: A meta-analytic review of the effects of financial
incentives on recovery after closed-head injury. The American Journal of Psychiatry, vol 153, no 1,
7- 10.
British Psychological Society (2009). Assessment of effort in clinical testing of cognitive functioning
for adults. British Psychological Society, Leicester.
Bush, S. et al (2005) Symptom validity assessment: Practice issues and medical necessity NAN
Policy and Planning Committee. Archives of Clinical Neuropsychology vol 20, 419-426.
Bush, S. (2005) Independent and court ordered forensic neuropsychological examinations: Official
statement of the National Academy of Neuropsychology. Archives of Clinical Neuropsychology,
vol 20, 997- 1007.
College of Alberta Psychologists (1987, revised 2005) Professional Guidelines for Psychologists: The
Control and use of tests by psychologists.
Eatwell, J. and Wilson, I. (2007). The effective use of psychometric assessments in decision making.
In Evans, I. et al (editors) Professional practice of psychology in Aotearoa New Zealand. Published
by the New Zealand Psychological Society, Wellington.
European Federation of Psychologists Associations and the European Association of Work and
Organisational Psychologists (2005) European Test User Standards for test use in Work and
Organisational Settings. Downloaded 8/03/2011 from: www.efpa.eu/professionaldevelopment/
tests-and-testing
Faleafa, M. (2009). Community rehabilitation outcomes across cultures following traumatic brain
injury. Pacific Health Dialogue vol 15, no 1, 28-34.
Fernando, K., Chard, L., Butcher, M. And McKay, C. (2003) Standardisation of the Rey Complex
Figure Test in New Zealand children and adolescents. New Zealand Journal of Psychology, vol 32,
no (1), 33-38.
Greenberg, S. and Shuman, D. (1997). Irreconcilable conflict between therapeutic and forensic
roles. Professional Psychology: Research and Practice vol 28, no 1, 50-57.
28
Houkamau, C. And Sibley, C. (2010) The multi-dimensional model of Māori identity and cultural
engagement. New Zealand Journal of Psychology, vol 39, no 1, 8-28.
Heilbronner, R. et al (2009) American Academy of Clinical Neuropsychology consensus
conference statement on the neuropsychological assessment of effort, response bias and
malingering. The Clinical Neuropsychologist, vol 23, no 7, 1093-1129.
Howe, L. et al (2010) Third Party Observation during neuropsychological evaluation: An update on
the Literature, Practical advice for practitioners, and future directions. The Clinical
Neuropsychologist, vol 24, no 3, 518-537.
International Test Commission (2005) International guidelines on computer-based and internet
delivered testing. Downloaded 8/03/2011 from www.intestcom.org
International Test Commission (2000) International Guidelines for Test Use. Downloaded 8/03/2011
from www.intestcom.org
Iverson, G. (2006) Ethical issues associated with the assessment of exaggeration, poor effort and
malingering. Applied Neuropsychology vol 13, No 2, 77-90
Iverson, G. (2007) Editorial: Identifying exaggeration and malingering. Pain Practice, vol 7, no 2,
94-102
Kaufman, P. (2009). Protecting raw data and psychological tests from wrongful disclosure: A
primer on the law and other persuasive strategies. The Clinical Neuropsychologist vol 23, no 7,
1130-1159.
Larrabee, G. (2012) Performance validity and symptom validity in neuropsychological assessment.
Journal of the International Neuropsychological Society, vol 18, 625-631.
Lavelle, E. and Barker-Collo, S. (2011) Neuropsychology and the Assessment of Competence. In
Seymour, F. et al (editors) Psychology and the Law in Aotearoa New Zealand. Published by the
New Zealand Psychological Society, Wellington.
Macy, M. & Bagnato, S.J. (2010). Keeping It "R-E-A-L" with Authentic Assessment. NHSA Dialog: A
Research-to-Practice Journal for the Early Intervention Field, 13, no1, 1-20
Ministry of Health. (2009). Operational guideline for the assessment of intellectual disability to
access disability support services contracted for people with intellectual disability in New Zealand.
O’Connor, F. (1993) Psychological Assessment Practice Guidelines. PA Consulting Group.
Ogden, J. et al (2003). Adapting neuropsychological assessments for minority groups: A study
comparing white and Maori New Zealanders. Brain Impairment, vol 4, no 2, 122-134.
Ogden, J. (2007). Practising clinical neuropsychology in Aotearoa New Zealand. In Evans, I. et al
(editors) Professional practice of psychology in Aotearoa New Zealand. Published by the New
Zealand Psychological Society, Wellington.
Pope, K.S. “Fallacies and Pitfalls in Psychology: 10 Fallacies in Psychological assessment”
Kspope.com/fallacies/assessment.php
Scott, K. et al (2000). A challenge to the cross-cultural validity of the SF-36 health survey: factor
structure in Maori, Pacific and New Zealand European ethnic groups. Social Science and
Medicine vol 51, 1655-1664
Slick, D. et al (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed
stands for clinical practice and research. The Clinical Neuropsychologist vol 13, no 4, 545-561.
29
Thomas, D. (1988) Development of a test of Māori knowledge. New Zealand Journal of
Psychology, vol 17, 59-67.
Wong, T. (2006) Ethical controversies in neuropsychological test selection, administration and
interpretation. Applied Neuropsychology vol 13, no 2, 68-76.