Assessment Tool Evaluation for Home Care in New York State

Objectives and Background

New York State manages a diverse array of programs designed to cater to the needs of individuals requiring long-term care. These programs support clients through various services, including:

• Skilled Nursing Facilities
• Home Care
• Hospice
• Assisted Living Programs
• Adult Day Health Care
• Adult Homes
• Managed Long Term Care
• Expanded In-Home Services for the Elderly Program

Over time, the state has developed and implemented more than thirteen different assessment tools across its agencies to effectively determine client needs and appropriate service options within these programs. Many of these tools were developed internally, and some have been in use for over a decade. This proliferation of assessment tools has led to inconsistencies across different programs, duplication of effort, and concerns about the validity and reliability of assessments in certain instances.

To address the complexities arising from the use of multiple assessment instruments across various programs, the New York Department of Health established a work group. This group was tasked with evaluating the feasibility of adopting a Uniform Assessment Tool (UDT) for use across all long-term care programs. The primary objectives for adopting a UDT included:

• Maintaining compliance with both New York State and Federal data set requirements.
• Ensuring applicability across all Long-Term Care settings, such as Skilled Nursing Facilities (SNF), Home Care (HC), Assisted Living Programs (ALP), Adult Day Health Care (ADHC), and the Expanded In-Home Service for the Elderly Program (EISEP).
• Facilitating electronic data collection and transmission.
• Reducing administrative burdens and streamlining care coordination processes.
• Gathering comprehensive health and medical information.
• Assessing functional needs and abilities through empirically validated and reliable methods.
• Evaluating the availability of informal support systems.
• Ensuring the quality, consistency, and completeness of both assessments and service plans.
• Guaranteeing adherence to CMS protocols and standards.
• Promoting consistency and objectivity in the application of state policies and procedures.
• Incorporating MDS (Minimum Data Set) and OASIS (Outcome and Assessment Information Set) quality indicators.
• Aiding in care planning and protective oversight.
• Enhancing payer accountability.
• Streamlining data collection and information gathering for policy development.
• Supporting personalized services that reflect the preferences and capabilities of individual consumers.

A robust and effective comprehensive assessment tool relies on the principles of standardization and validation. In well-developed instruments, questions and embedded assessment scales are standardized and rigorously validated. Standardization and validation are critical for preventing common issues and inconsistencies when applying the assessment tool to diverse clients and care settings. A significant number of the existing thirteen-plus data sets and assessment tools in use in New York State lack adequate standardization and have not undergone thorough reliability or validity testing. Furthermore, many of these tools are not supported by decision-support algorithms that could assist in Long-Term Care assessment and performance evaluation.

Consequently, the development of care plans, which form the basis for Long-Term Care referrals, often becomes subjective and influenced by the inconsistencies inherent in the assessment tool used. The negative outcomes of these subjective decisions can range from inefficient allocation of limited resources and inappropriate service utilization to administrative redundancies and obstacles in care transitions across different settings or regions. Moreover, the absence of MDS-based tools prevents policymakers from effectively comparing the characteristics of home and community-based populations with those in nursing home settings. Standardized assessment is particularly crucial for programs aimed at nursing home diversion, making a uniform Assessment Tool For Home Care especially important.

The Office of Long Term Care (OLTC) undertook an extensive review of research literature to identify Uniform Data Set (UDS) models and templates that have been developed and utilized by other states and countries. The primary goal of these reviews was to find potential tools that could bring uniformity and validity to the information used within New York’s Long-Term Care service system. This effort identified several promising tools for use as Uniform Assessment collection instruments, including InterRAI, the CARE tool (Continuity Assessment Record and Evaluation) under development and pilot testing by CMS, and the SAAM tool. Following an internal evaluation of these tools by OLTC, FOX Systems Inc. was contracted to conduct an independent review and validation of the internal analysis, identify any additional considerations, and explore alternative tools that could enhance assessments. Finally, FOX Systems was tasked with comparing the candidate UDTs with the assessment instruments currently employed in New York State. After reviewing the existing work, FOX Systems conducted a goodness-of-fit and gap analysis, comparing the current New York State tools with the potential UDT replacements.

Scope of Work

To accurately determine an individual’s capacity to remain at home and to identify the essential support services necessary for a safe home environment, a complete and reliable home assessment is indispensable. Consistency in assessment data, processes, and professional judgment is paramount for the efficient and timely documentation of required information.

The initial step in this process involved developing an “Ideal” domain assessment framework. This domain set was designed to encompass the broad spectrum of medical, mental health, social, functional, and personal needs of an individual. In formulating these ideal domains, careful consideration was given to the diverse settings in which the assessment tool might be applied. FOX Systems staff collaborated closely with OLTC personnel to establish a consensus on a set of domains and high-level assessment topics that would comprehensively represent the full range of client needs.

The development of this ideal domain set was informed by multiple sources, including:

• A thorough literature review aimed at capturing the most up-to-date information on comprehensive assessment practices.
• An examination of existing assessment tools used within New York State.
• A review of tools developed by the Centers for Medicare & Medicaid Services (CMS).
• An analysis of current national and international assessment tools.

This analysis resulted in the identification of twelve distinct domain sets and the specification of high-level data points to be captured within each. The detailed domain set is presented in Appendix A.

Direct comparisons between the ideal domain set and the assessment categories used by New York State staff are complex due to variations in tool organization, which can create superficial differences. For example, the ideal domain set includes a domain called “Environmental Conditions,” which incorporates many elements found in the New York domain of “Social History.” After accounting for these naming conventions and organizational differences, two additional domains were identified as essential for the New York State set:

• Prevention
• Personal Preferences

A critical aspect in evaluating a sophisticated comprehensive assessment tool is the inclusion of uniform and validated questions that measure function, cognition, behavior, and mood. These standardized measures ensure consistency in results over time and across different care settings. Both InterRAI and CARE tools are largely constructed upon validated embedded assessment scales. A key consideration in evaluating the fourteen assessment tools against our ideal domain set was the presence and utilization of these validated embedded assessment scales in each tool. Our findings indicate where a domain was addressed, but an embedded assessment scale was not employed, highlighting potential areas for improvement in assessment standardization for home care.

Process

The project team began by identifying and defining the essential domains for a person-centered home care assessment. For each domain, staff analyzed and listed the required elements. The resulting domain assessment summary outlines the information needed to conduct a thorough home care screening and to initiate a safe and practical plan of care. This summary documents twelve major domains, which were then used as the framework for a detailed review of the identified assessment tools. The domains and their components are fully detailed in Appendix A.

Evaluation

Project staff conducted an evaluation of four assessment systems that are widely used nationally or internationally, focusing on their suitability as an assessment tool for home care. Each tool was reviewed to determine the extent to which it addressed the essential assessment domains previously defined. The results of these evaluations are summarized in Table 1: Domain Evaluation Summary. Additionally, fourteen Long-Term Care Assessment tools currently in use within New York State were evaluated against the same ideal domain set. The comparative results are also presented in TABLE 1: DOMAIN EVALUATION SUMMARY.

FOX Systems performed a gap analysis to pinpoint weaknesses in the evaluated assessment tools when compared against our ideal domain set. This analysis included fourteen New York State tools, three national tools, and one international tool. A domain was considered “validated” if the tool adequately addressed the essential components of that domain. If the essential components were not met or were absent, the tool was rated as “not meeting” the domain criteria. In instances where a domain was partially addressed but with significant shortcomings, it was categorized as “partially met,” and detailed comments were recorded to explain the qualifications. The strengths and limitations of each tool were also documented in the comment section to provide a nuanced understanding of each tool’s capabilities.

FOX Systems concluded that no additional assessment tools beyond those already considered would significantly enhance the capabilities of a Uniform Assessment Tool for home care. The OLTC staff had already identified all potential candidates that could serve as a basis for a UDT, and these were comprehensively included in our analysis.

Findings

The detailed results of the assessment tool analysis are presented in Table 1. This table lists the evaluated tools in the first column. General-use tools employed in multiple states are listed first, followed by the tools currently used within New York State. The domains against which each tool was evaluated are listed in the first row of each section. Appendix A provides a more detailed description of the specific categories used to evaluate each domain.

Each tool received a score for each domain. In Table 1, a filled circle (●) indicates that the tool adequately meets our requirements for that domain. An empty circle (○) signifies that the tool does not adequately address the domain requirements. If a tool partially met the requirements, a comment reference number is indicated, and detailed comments and concerns are provided in the comments column on the far right of the table.

Each domain that is not met or is only partially met with qualifications represents a gap between the tool’s assessment capabilities and the comprehensive set of capabilities required by New York State for effective home care assessment. The ideal UDT selection should minimize or eliminate these identified gaps. However, as indicated in Table 1, no single tool currently available completely fulfills the requirements for all domains without the need for supplementary data collection elements.

Key highlights from the findings are summarized below:

New York State Tools

Ten assessment tools currently utilized by New York State were evaluated. Overall, these tools do not comprehensively cover more than seven of the twelve domains in our essential domains listing. The substantial number of empty circles and commented cells in the evaluation matrix clearly indicates this lack of complete domain coverage. Furthermore, none of the reviewed tools would meet the criteria of a mature, standardized, comprehensive evaluation tool because all lack validated, embedded scales in one or more critical domains. Specific problems identified include:

• The frequent use of multiple tools to assess a single client, leading to the redundant collection of certain information.
• The use of some tools primarily for eligibility determination rather than a comprehensive evaluation of social needs.
• A general lack of standardized tools or scales within most tools, resulting in inconsistent data collection practices.
• The prevalence of simple yes/no questions in some tools, which limits the ability to thoroughly document the underlying needs of individuals.
• The absence of MDS 2.0 incorporation in any tool, preventing meaningful comparisons of client populations to nursing home populations.
• The inclusion of lengthy narrative sections in some tools, which introduces potential for bias and incompleteness in evaluations and hinders automated comparisons across clients or client groups.

General Use Tools

Four tools designed for general use were reviewed, although the CARE tool was still in the pilot phase at the time of evaluation.

MDS 3.0

The Minimum Data Set (MDS 3.0), mandated by CMS for use in Skilled Nursing Facilities across the U.S., was evaluated based on the version slated for release in 2010. MDS 3.0 incorporates assessment of personal preferences and demonstrates enhanced sensitivity in certain domain assessments. However, even with these improvements, it still lacks detailed information regarding demographics, medication management, and home living situations, which are crucial aspects of a comprehensive assessment tool for home care. It does integrate valuable components such as the Brief Interview for Mental Status (BIMS), Confusion Assessment Method (CAM), and Patient Health Questionnaire-9 (PHQ-9) for mental health evaluation. These components offer improved sensitivity compared to the MDS 2.0 tool. MDS 3.0 also shows closer alignment with selected OASIS measures compared to its predecessor.

OASIS

OASIS (Outcome and Assessment Information Set), also mandated by CMS for home health care nationwide, was assessed against our defined domain set. The OASIS C Tool was evaluated using the same domains as presented in Table I. OASIS C adequately covers most domains and meets the requirements for all except the preventative and personal preferences sections. It provides a comprehensive behavioral and functional status section. However, the OASIS tool does not incorporate validated embedded scales, resulting in a lack of alignment with MDS 3.0 scales. Consequently, OASIS is not considered a comprehensive assessment tool capable of stand-alone use and is therefore not a primary candidate for a UDT.

CARE

CARE (Continuity Assessment Record and Evaluation) Tool – Home Care version was evaluated against our domain set. Developed under the direction of CMS as part of the Post-Acute Care Payment Reform Demonstration, CARE is undergoing pilot testing in 10 states for Post-Acute Care and care transition projects. As it was still in the demonstration phase and not yet fully implemented, it presented both opportunities and limitations as an assessment tool for home care. The CARE tool met all domains except for preventative and personal preferences sections. It demonstrates strong alignment with MDS 3.0 and includes certain OASIS C items. It also incorporates standardized assessment instruments like the Brief Interview for Mental Status (BIMS) and Patient Health Questionnaire (PHQ-9) for mental health evaluations. Given its ongoing development, potential revisions before finalization could address the identified weaknesses or missing domains, particularly in areas relevant to home care populations. CMS has expressed interest in incorporating domains more specifically focused on home care needs in the final version.

InterRAI

The International Resident Assessment Instrument (InterRAI) Tool – Home Care version is utilized in several U.S. states for waiver populations, including New Jersey, Michigan, Massachusetts, and Louisiana, as well as in other countries across Canada, Europe, and Japan. Developed through international collaboration, InterRAI aims to evaluate outcomes and needs of individuals across the continuum of care settings. The InterRAI tool is based on the MDS 2.0 assessment instrument, and current plans do not indicate modifications to align it with MDS 3.0. The InterRAI assessment tool met all domains with the exception of the personal preference section, making it a strong candidate as an assessment tool for home care.

The InterRAI tool integrates scored measurements throughout all domains. The Cognitive Performance Scale (CPS) and Depression Rating Scale (DRS) questions are incorporated to assess mental health status. The instrumental activities of daily living (IADL) function section evaluates both performance and capacity using a numeric scale.

The InterRAI tool has undergone extensive testing and scientific validation and is currently in use in numerous states and countries, establishing it as a stable and mature assessment instrument for home care and long-term care needs.

Conclusion

Upon final analysis, no single assessment tool perfectly met all of the established domain criteria. However, the national and international tools, particularly InterRAI and CARE, significantly outperformed the New York State tools, meeting a much higher proportion of the domain criteria. The InterRAI tool met the highest number of domains, closely followed by the CARE tool, which scored well in all domains except prevention. Notably, the MDS 3.0 tool was the only national and international tool to satisfy the domain requirements related to personal preferences, a critical criterion for NYS UDT candidates and essential for person-centered home care.

None of the assessment tools currently used in New York State adequately measured as many of the domains as either the InterRAI or the CARE tool. Many of the New York tools are limited in scope, primarily oriented towards specific data elements, and often focused on a subset of care settings. None of the New York tools evaluated met more than seven of the thirteen domains. Critically, none of the current New York tools met the definition of a mature comprehensive assessment instrument with validated scales to standardize assessment results, a key feature needed for a reliable assessment tool for home care.

The primary conclusion from this analysis is that New York State has a significant opportunity to enhance the scope and effectiveness of its assessment processes by adopting one of the national or international tools identified as potential UDT candidates. The InterRAI and CARE tools emerge as the strongest candidates among the tools evaluated. While the CARE tool was still under development until 2010, this also presents an advantage. The delay in finalization meant the tool could be further modified to better reflect the specific needs of the in-home population, potentially making it an even more tailored assessment tool for home care. CMS expressed a willingness to collaborate with states to develop these additional capabilities. New York State could potentially join the demonstration phase by 2010 to contribute an in-home care perspective to the tool’s refinement.

For New York State to successfully adopt a uniform assessment process and achieve better integration of its long-term care programs, a valid and comprehensive UDT is indispensable. As this analysis demonstrates, current tools lack the breadth of data required for a comprehensive assessment of clients across all programs. Both the InterRAI and the CMS CARE tool offer a complete assessment tool set, along with the validation and reliability that are currently lacking in New York State tools. The ultimate choice between these leading candidates will depend on further evaluation based on additional criteria and specific implementation needs.

Appendix A: Domain Assessment

Essential Components of a Comprehensive Needs Assessment, 03/24/2009

1. Demographics

• Multiple examples available
• Basics (such as address, date of birth)

Consider

• Primary language/need for interpreter
• Literacy level

Children’s assessment includes school history

• Home schooled
• Mainstream schooling history
• Special program needs

2. Disease Process / Risk Factors

Active/Current Disease Process

• Complete list including treating physician
• Does person have a neurological diagnosis?
• Does person have a history of seizures? Count of incidents from the previous year

Chronic Disease Processes with date of onset

• Diabetes
• COPD
• CHF
• Heart Disease
• Congenital Disorder

Treatments and therapies for identified conditions

Allergies with cause and reaction

Risk Factors

• Smoking
• Weight issues
• Alcohol use
• Drug use
• Shortness of breath with activity

Fall Risk Assessment

• Verify if person has had two or more falls in last year
• Verify if person has had any fall with injury in the last year

3. Mental Health

• Cognition
• Memory
  • Long-term
  • Short-term
  • Orientation
    • Person
    • Place
    • Time
• Comprehension/Skill with problem solving
  • Complex ideas
  • Abstract ideas
  • Basic daily needs
• Social skills
• Mood
  • Depression
  • Anxiety
• Behavior
  • Wandering behavior
  • Verbal disruptions
  • Physical aggression
  • Inappropriate demonstrations

Observe for triggers for abuse or intentional injury

4. Function

• Basic activities of daily living (ADLs)
  • Bathing
    • Shower
    • Tub bath
    • Sponge bath
  • Grooming
    • Oral care
    • Hair grooming
    • Shaving
    • Make-up application
  • Dressing and undressing (include use of prosthesis or orthosis)
  • Toileting
  • Transferring
  • Continence of bowel and bladder
  • Feeding (eating/swallowing issues/ nutrition)
    • Regular diet
    • Modified consistency
    • Tube/Parenteral feeding
    • Hydration concerns

(Based on Katz Basic ADL Scale)

• Instrumental activities of daily living (I ADLs)
  • Ability to use standard telephone or cell phone
  • Shopping for groceries and clothing
  • Food preparation
    • Cooks for self
    • Caregiver prepares meals
    • Meals on Wheels
  • Housework
  • Laundry
  • Mode of transportation
    • Drives self
    • Driver required
    • Private car or van
    • Adaptive equipment required
    • Taxi
    • Public transportation
  • Management of medications
  • Ability to handle own finances
  • Sleep habits
    • Number of hours per night
    • Napping
    • Location
      • Standard bed
      • Special medical bed
      • Recliner
      • Other
    • Use of Bi-Pap or C-Pap at night

(Based on Lawton-Brody IADL scale)

5. Communication (Sensory Status)

• Evaluation of hearing, vision, and speech
• Adaptive devices and date last evaluated
  • Glasses/contacts
  • Hearing aids
  • Dentures
• Verify the following
  • Ability to hear normal conversation and electronics
  • Ability to see in normal lighting
  • Ability to clearly express ideas and needs
  • Ability to use standard telephone or cell phone

6. Mobility

• Ambulation (include use of assistive devices)
  • Cane
  • Walker
  • Wheelchair
  • Prosthetic device
• Locomotion
  • Walking
  • Wheelchair use (if needed)
  • Stair climbing ability (1 flight is 12-14 steps)

7. Environmental (Living Conditions)

• Status
  • Lives alone
  • Lives with Caregiver
    • Spouse
    • Parent
    • Child
    • Relative
    • Paid caregiver
    • Other
• Hours of caregiver time required
  • 24/7 supervision required
  • Several times a day (am and pm)
  • One visit daily
• Structural Concerns
  • Uneven surfaces in travel path
  • Stairs
  • Hallways
  • Doorways
  • Space limits
  • No locks on exterior doors
  • Poor temperature control
• Risks Observed
  • No working toileting facilities
  • No safety devices in specialized areas
  • No cooking facilities
  • No refrigeration facilities
  • Inadequate lighting
  • No running water
• Working communication device
  • Land line telephone
  • Cell phone
  • Medical alert system
  • Web based monitoring system

8. Medication Management

• Oral medications (including all over the counter medications)
  • Name
  • Dose
  • Route
  • Reason
  • Prescriber
• Self Management Issues
  • Pharmacy pick-up
  • Delivered
  • Mail order
  • Requires weekly set-up
• Injected
  • Self-administered
  • Assistance required (lay or professional)
• Inhalants (Hand held inhaler or nebulizer)
  • Self administered
  • Assistance required (lay or professional)

9. Skin Integrity

• Is skin currently intact? (Note any skin concerns)
• Verify Braden and PUSH Tools are used
• Document history of pressure ulcers
• If a wound or ulcer is present, list the location and character of each area (grid)
• Wounds/Ulcers verification
  • Number of pressure ulcers
  • Number of stasis ulcer
  • Number of non-healed surgical wounds
  • Number of other wounds or injuries
• Verify feet condition and treatment (if applicable)

10. Pain Management

• Pain presence frequency
  • None
  • Rarely
  • Daily
  • Weekly
• Pain severity (scale 0 to 10)
• Pain effect on
  • ADLs
  • Sleep
  • Mood
• What provides relief?
  • Medication schedule
  • Alternative methods tried

11. Prevention Services

• Verify dates of
  • Last complete physical
  • Flu immunization
  • Pneumococcal immunization
  • Tetanus immunization
  • Mammogram
  • Colonoscopy

12. Personal Preferences Section

• Measurement Verify a section is available for preferences such as
  • Chooses to work, if able
  • Choosing clothes to wear
  • Caring for personal belongings
  • Reading books, newspapers, or magazines
  • Bathing preference (tub, shower, or sponge)
  • Listening to music
  • Receiving shower
  • Being around animals such as pets
  • Keeping up with the news
  • Doing things with groups of people
  • Snacks between meals
  • Participating in favorite activities
  • Staying up past 8:00 p.m.
  • Spending time away from the house
  • Spending time with family or significant other
  • Spending time outdoors
  • Involvement in care discussions
  • Participating in religious activities or practices
  • Place to lock personal belongings
  • Use of telephone in private