Methodological Details

Measures were included in the IBHC Measures Atlas based on a series of expert reviews.

(1)  A team member reviewed the titles and abstracts of all search results and eliminated sources that did not have clear relevance to the definition of integrated behavioral health care or health care quality measurement.

(2)  A panel of experts then reviewed the full text of each measure to assess relevance to the functional domain of the Integration Framework. Measures initially considered relevant were passed along for mapping to the Framework; measures that were not relevant were eliminated.

A team of reviewers engaged in an iterative process of mapping the remaining measures to the measurement constructs of the Integration Framework. Reviewers worked consecutively and had access to prior reviewers’ mapping decisions. Discrepancies were resolved through discussions among the reviewers. When reviewers disagreed with the expert panel regarding inclusion of a measure in the IBHC Measures Atlas, the measure was re-presented to the expert panel for a final decision.

Following the mapping of the measures, reviewers conducted a final usability and usefulness check in which they assessed how well each measure aligned with the measurement constructs, the breadth of constructs addressed by a given measure, and the burden to complete the measure. Results of the usability and usefulness check were used to divide the final 17 measures into two categories. See Overview of Measures included in the IBHC Measures Atlas, earlier in this document, for the full list of measures.

The purpose of the literature search was to find peer reviewed articles and/or reports that focused on the quality of integration between behavioral healthcare and physical healthcare settings (e.g. Integrated Behavioral Health Care (IBHC)). There are multiple dimensions to consider when assessing health care quality. For this literature search project, we are using clinical health outcomes as the primary markers of quality. The literature search was limited to prevalent mental and physical health conditions that are commonly comorbid. The physical health conditions included: diabetes, cardiovascular disease, high blood pressure, osteoarthritis, and rheumatoid arthritis. The mental health conditions were depression, anxiety disorders, and substance abuse.

A three phase search strategy was used. First, staff from Westat’s IR Center conducted an extensive search for clinical outcome measures that assessed the quality of integration in healthcare settings. Initial searches were conducted in the following resources:

• National Quality Measures Clearinghouse. Covers evidence-based quality measures and measure sets, including health care delivery measures and population health measures.
• Applied Social Sciences Index and Abstracts (ASSIA). Covers health, social services, psychology, sociology, economics, politics, race relations, and education literature.
• Health and Psychosocial Instruments (HAPI). Provides information on measurement instruments (i.e., questionnaires, rating scales, tests, index measures, coding schemes, checklists) in the fields of public health and medicine, psychosocial sciences, communication, organizational behavior, and others. HAPI records do not contain the actual instruments, but often direct the researcher to sources of measures needed for research, assessment, and evaluation.
• PsycINFO. Covers international literature in the psychological, psychiatric, social, behavioral, and health sciences.
• PUBMED. Covers all aspects of health and medical literature.

Results were limited to English language and to articles or reports published between 2012 and 2013. The general search was an update of a search conducted in 2012, so the choice of publication dates was designed with an overlap of 1 year, in order to be certain of coverage. 

After the general literature search that updated the 2012 work, two additional searches were conducted covering financial outcomes and provider experience with integrated behavioral health care. These two literature searches included documents published between 2001 and 2013. 

The Financial Outcomes search focused on the costs of planning and delivering integrated behavioral health care from the perspective of health care provider organizations, the cost of receiving integrated behavioral health care from the patients’ perspective, and health care utilization costs for patients receiving integrated behavioral health care from the payers’ perspective (public and private insurers).

The Providers search focused on provider experience with the structures and processes involved in implementation of integrated behavioral health care to patients; the perceived changes in culture and professional identity, experience working in an integrated setting; and the quality of care that results from implementation of integrated behavioral health care strategies. Provider was broadly defined as including: physician, nurse, nurse practitioner, physician assistant, psychologist, behavioral therapist, nutritionist, counselor, and therapist.

The terms used in the searches appear in the table below. Terms were combined with Boolean operators (i.e., AND, OR connectors). Various combinations of terms were searched and citation titles and abstracts were scanned to determine if the content related to measures of integrated behavioral health care in the focus areas noted above. The results from this search identified several articles.

During the second phase of our search strategy, we performed an initial review of the articles from phase one. The results indicated no specific clinical outcome measures for evaluating quality of integrated care existed. Measures in the articles either focused on an individual behavioral health condition or the physical health outcome alone. Several measures focused on the identification of specific health conditions (e.g. screening), tracking symptoms, patient experience, and processes of care. Measuring the impact of integrated care on clinical outcomes requires an innovative approach where we use a collection of measures to evaluate different dimensions of the clinical care experience. The team completed a thorough review of the individual measures and the following selection criteria were used: administrative ease for the PCP/ staff, publicly available, commonly used in research, tool to track improvement over time, application to multiple framework domains, and cultural sensitivity/used in diverse populations. The number of promising measures was limited.

The third phase of the literature search required the identification of additional articles on the psychometric properties of the outcomes measures that are based on survey instruments. The IR Center identified the primary source document for each of the proposed measures. The IR Center librarians used HAPI (Health and Psychosocial Instruments), PsycInfo, and PubMed databases to search for the measure names or acronyms.

We evaluated the quality of the measures based on the following factors: sensitivity, specificity, validity, and reliability. Given the limited information on some of these criteria in the articles, we assessed the psychometric properties as available. In order to be selected for the final list of clinical outcome measures, the measure was required to have at least 2 good psychometric properties. From the literature search results, many of the commonly used clinical outcome measures for physical health conditions are not based on brief survey instrument data like the ones for depression, anxiety, or health-related quality of life. Nonetheless, conventional clinical outcomes like controlled blood glucose for Type II diabetes are equally important for assessing quality of integrated care for patients.

Key definitions: 
Sensitivity is the ability of a test to correctly classify an individual as ‘with health condition’.
Specificity is the ability of a test to correctly classify an individual as ‘health condition-free’.
Reliability is the extent to which a measure produces consistent results under various conditions.
Content validity refers to the extent to which a measure represents all dimensions of the construct (e.g., personality, ability) that is being measured.