Introduction
The global population is aging rapidly, with projections indicating a doubling of individuals aged 65 and over between 2016 and 2050 [1, 2]. This demographic shift brings into sharp focus the increasing prevalence of frailty, a condition affecting approximately a quarter of older adults [3, 4]. Frailty is characterized by a decline in physiological reserves, rendering individuals more vulnerable to stressors and hindering their ability to recover from acute health challenges [5]. This heightened vulnerability significantly elevates the risk of adverse health outcomes, including hospitalization and mortality [6, 7]. Therefore, accurate and timely actual frailty assessment for care planning is becoming increasingly critical in healthcare systems worldwide.
Recognizing frailty early is crucial, and numerous screening tools have been developed, such as the Frailty Phenotype and the Clinical Frailty Scale (CFS) [8–12]. Among these, the Frailty Index (FI), based on the accumulation of health deficits, stands out for its ability to quantify frailty levels. This method acknowledges that as the number of health deficits increases, so does an individual’s frailty level [13–15]. The FI has been extensively validated across diverse countries and healthcare settings, solidifying its reliability and applicability [5, 16–21].
A particularly valuable approach to constructing an FI is through the utilization of data from a Comprehensive Geriatric Assessment (CGA) [22–24]. The CGA is a holistic evaluation of older adults’ health, encompassing cognitive, emotional, motivational, sensory (vision, hearing, speech), sleep, pain, musculoskeletal (strength, balance, mobility), functional (activities of daily living), social, medication, and psychological (control of life, health attitude) aspects. By providing a comprehensive understanding of an individual’s circumstances and potential impairments, the CGA enables healthcare professionals to develop holistic, patient-centered care plans tailored to complex needs [25–28].
The Frailty Index derived from CGA data (FI-CGA) was initially developed using data from the Canadian Study of Health and Aging [22, 23]. It has demonstrated strong correlation with generic FIs [22–24] and predictive capabilities for outcomes like hospital length of stay and discharge destination. Furthermore, its validity has been confirmed using health evaluations completed by caregivers [29–34]. Despite the clear clinical benefits of actual frailty assessment for care planning using FI-CGA, traditional paper-based methods of data collection and manual processing have hindered its widespread adoption. These methods are often time-consuming and prone to errors, presenting significant barriers for busy frontline clinicians.
The emergence of electronic frailty assessments has gained considerable momentum in recent years. Researchers in the UK pioneered the use of electronic health records to calculate an electronic FI (eFI) for millions of patients [35, 36]. This eFI approach has been rapidly adopted and adapted in countries like Scotland, Australia, and Canada, leveraging large primary care datasets extracted from Electronic Medical Records (EMR) to automate eFI calculations [37–39]. Subsequent studies have further explored and validated the eFI’s features and the use of routine data in frailty assessment for both aging research and clinical practice [40–44]. The eFI has proven to accelerate frailty screening and consistently demonstrates that higher eFI scores correlate with increased risks of mortality, hospitalization, and institutionalization [35, 36].
However, the eFI is often regarded as a screening tool requiring “eyes-on” validation for individualized patient care decisions. This highlights the potential of developing an electronic FI-CGA (eFI-CGA) that simultaneously conducts a CGA and automates the FI calculation. Such a tool could offer a more efficient, time-saving, and cost-effective approach to actual frailty assessment for care planning in clinical settings, ultimately enhancing integrated healthcare for older patients.
Driven by this need, we developed a Standalone eFI-CGA software tool, an electronic version of the CGA (eCGA), designed for frontline care providers to perform CGAs and automatically generate eFI-CGA scores on personal computers. Our goal was to facilitate the integration of eFI-CGA into EMR systems, making actual frailty assessment for care planning more accessible and transparent. Our specific objectives were: 1) to design and implement an eCGA form mirroring a widely used paper-based CGA form; 2) to ensure the accuracy, robustness, and efficiency of the eFI-CGA calculation; and 3) to optimize the eFI-CGA software tool for convenient clinical use. This innovative software tool is now ready for use by frontline healthcare providers, enabling both computerized CGA evaluation and automated FI-CGA calculation, thereby improving actual frailty assessment for care planning.
Methods
Operating System, Platform, and Programming Language
When developing the eFI-CGA application, our initial considerations centered on the operating system (OS), supporting software, and hardware commonly accessible to our primary end-users: healthcare professionals such as family physicians, geriatricians, and emergency doctors. Given the widespread availability of Microsoft Windows computers in healthcare settings, we designed the Standalone eFI-CGA application to operate on Microsoft Windows 2000 or later.
Considering the stringent security protocols on many healthcare Windows devices, which often restrict the installation of additional applications due to patient privacy concerns, we aimed for an application that could function using the inherent Microsoft Windows libraries, minimizing reliance on external third-party components. Among the Windows .NET libraries suitable for client application development, Windows Forms (WinForms) and Windows Presentation Foundation (WPF) are prominent. While WPF offers greater flexibility in user interface design, its dependence on .NET Framework 3.0 or later versions (https://docs.microsoft.com/en-us/dotnet/framework/winforms/) posed a compatibility challenge with older systems running Windows 2000. Therefore, we opted for WinForms, a platform known for rapid application development and compatibility with a broad range of devices supported by Microsoft Windows 2000 and .NET Framework 2.0 or later (https://docs.microsoft.com/en-us/windows/desktop/choose-your-technology).
The eFI-CGA application was coded using C# programming language. C# is a robust, general-purpose, object-oriented language well-suited for .NET application development. Microsoft’s implementation of C# is deeply integrated within the .NET framework, ensuring consistency between the C# and .NET object models. The abstraction provided by .NET simplifies the creation of complex user interface elements (like selection buttons and dropdown lists), facilitating the convenient development of Windows desktop applications using C# (https://docs.microsoft.com/en-us/dotnet/csharp).
User Requirements Analysis
Throughout the eFI-CGA application development, we prioritized a thorough analysis and fulfillment of key user requirements, detailed below, to ensure effective actual frailty assessment for care planning.
General Appearance of the eCGA Form
A primary user requirement was that the electronic CGA form should closely resemble a familiar, well-designed paper-based CGA form. This familiarity is crucial for clinicians, ensuring ease of use and minimizing disruption to established clinical workflows. We chose to emulate the Capital Health CD0184MR_06_09 form (Supplementary Materials #1), a widely adopted form, including its use in a randomized controlled trial assessing CGA in community-dwelling older adults in rural settings [45]. A consistent user interface promotes clinically meaningful evaluations and simplifies form completion, mitigating potential time constraints and minimizing the learning curve associated with new computer skills for end-users. To meet this requirement, we meticulously positioned virtually every item in the electronic form to mirror its location in the paper-based template (Figure 1).
Figure 1.
The main interface of the eFI-CGA application showcasing the electronic Comprehensive Geriatric Assessment form (eCGA). This digital form replicates a widely-used CGA template and includes all standard assessment items. Key features highlighted include “Preview,” “User Manual,” “Reset,” “Save Records,” and “Calculate eFI-CGA” buttons, along with examples of critical eCGA items such as “Clinical Frailty Scale,” “Number of Problems,” and “Medications.” The interface also demonstrates the flagging option for items requiring follow-up in the care plan and the variety of input methods available to the user.
Control for User Input
Another critical user requirement was the implementation of diverse field types to accommodate the form’s varied functions. Effectively addressing this requirement is essential to fully leverage the advantages of electronic methods over traditional paper-based forms, enabling robust input control for actual frailty assessment for care planning. To satisfy this, we incorporated four primary input methods tailored to specific fields (Figure 1): check boxes, radio buttons, text boxes, and drop-down lists. Check boxes were used for item selections requiring simple yes/no or present/absent choices. Radio buttons were employed to facilitate value selections within grouped options, ensuring that only one option could be selected per item at any given time. Text boxes were designed with context-specific constraints, such as restricting the “Age” field to accept only integer inputs. Drop-down lists provided a selection of predetermined options from a concise list. For instance, when entering cognitive assessment scores, users can choose the appropriate value from a predefined list of possible scores. Drop-down lists also enabled categorization of inputs (e.g., occupation), limiting response ranges and streamlining data processing.
Clinical Follow-Up Actions
The ability to flag items requiring clinical follow-up is vital for actual frailty assessment for care planning. This feature allows end-users to proactively plan patient care, directing physician attention to specific areas needing management. To meet this requirement, we integrated additional check boxes alongside a blank note-space. Users can select multiple check boxes to indicate several fields requiring follow-up and use the note-space to record detailed notes. This functionality directly supports the translation of assessment findings into actionable care plans.
User Interface Design
To ensure optimal functionality and a superior user experience, the graphical user interface (GUI) was designed with a three-page structure, each page featuring a clear and simple layout, a manageable number of actions, and a specific functional focus (Figures 1, 2). This streamlined approach enhances usability for clinicians during actual frailty assessment for care planning.
Page 1
This initial page was designed to display acknowledgments of use and emphasize the non-commercial nature of the application (Figure 2). Users are presented with terms of use and must explicitly “Accept” these terms to proceed with the assessment. Selecting “Decline” will close the application. This page serves as an important step in ensuring users are aware of the software’s purpose and usage restrictions.
Page 2
This page is straightforward, designed to collect essential background and demographic information for both the patient (e.g., healthcare number, age, gender, education) and the clinician (e.g., clinician identification) necessary for effective data management and actual frailty assessment for care planning tracking (Figure 2). All fields on this page must be completed before the user can proceed to the main eCGA form on Page 3. This ensures that essential identifying information is captured before the assessment begins.
Page 3
This is the central page of the application, housing the eCGA form and the eFI-CGA calculator (Figure 1). Users can navigate through the form sequentially, completing each item, and then click the “Calculate eFI-CGA” button at the bottom to generate the eFI-CGA score, facilitating actual frailty assessment for care planning. Page 3 also includes additional features such as “Save Records,” which saves user input in various formats (detailed in the Data Storage section), and “Reset,” which saves current inputs, clears the form, and navigates the user back to Page 2.
Pop-Up Pages
Page 3 also incorporates pop-up features to enhance user interaction. The “Preview” button generates a single-page image of the entire form in its current state, allowing users to review data trends and identify potential input errors before finalizing the assessment. The “User Manual” button opens a comprehensive user manual in PDF format, providing detailed instructions on eFI-CGA software usage and item-by-item guidance for completing the eCGA form, ensuring users have readily available support for actual frailty assessment for care planning.
Figure 2.
Page 1 and Page 2 of the eFI-CGA application. The top panel displays Page 1, which includes the terms and conditions of use for the application. The bottom panel shows Page 2, designed to collect essential background and demographic data for patients and clinicians, crucial for data management and effective actual frailty assessment for care planning.
Dynamic User Guidance
To dynamically guide users through the eCGA form completion and ensure accurate actual frailty assessment for care planning, we implemented several user guidance features. These include visual cues like colored reminders, error messages, and warning signs. For example, an error message is displayed if a user attempts to calculate the eFI score before completing all required assessment items (Figure 3). Additionally, colored reminders appear next to any specific fields that have been missed, prompting the user to complete them. These dynamic guidance features are crucial for minimizing errors and ensuring complete and accurate data entry.
Figure 3.
Examples of error and warning messages integrated into the eFI-CGA application to guide user interaction. The top section illustrates an error message that appears if a user tries to calculate the eFI-CGA score with less than 80% of the form completed. The middle section shows a pop-up error message triggered when a user attempts to proceed without filling in all mandatory fields. The bottom section displays visual symbols used to remind users to complete specific fields, ensuring thorough actual frailty assessment for care planning data collection.
Calculation of the eFI-CGA
The eFI-CGA calculation within the software adheres to the established deficit-accumulation-based FI approach. As represented in the formula below, frailty is quantified as the proportion of health deficits an individual has accumulated relative to the total number of health deficits under evaluation [12–16]. This calculation is central to providing an actual frailty assessment for care planning.
| [eFI-CGA]= ∑i=1nxin |
|---|
Following a pre-defined FI-CGA variable coding scheme (Supplementary Materials #2), each raw input value from the eCGA form is initially mapped to a coded value ranging between 0 and 1. Values closer to 1 signify a greater risk or deficit. These coded values are then stored in a list data structure for processing. For instance, if a patient’s “Motivation” is assessed as “High,” “Usual,” or “Low,” the corresponding coded values “0,” “0.5,” and “1” are added to the list, respectively. The comorbidity count question (“Number of Problems”) is also incorporated into the deficit count for the eFI-CGA calculation. To mitigate a ceiling effect, the raw input value of the comorbidity count is included in the deficit count (up to a maximum of 18, even if the actual count exceeds this), and the total number of deficits under evaluation also includes a maximum of 18 deficits from comorbidities. This capping at 18 balances the ratio of comorbidity diagnoses to the total deficits considered (14 in this CGA form), which is consistent with previously studied community and clinical datasets where comorbidities were recorded in a text field. The coded values and the comorbidity count, totaling n deficits, are used to compute the eFI-CGA score using the formula above. The eFI-CGA score is calculated with a precision of four decimal places, providing a granular measure for actual frailty assessment for care planning.
Consistent with standard FI construction procedures [46], an 80% non-missing value threshold is implemented. The eFI-CGA score is only calculated if at least 80% of the required fields in the eCGA form are completed. The system checks for this completion threshold before generating a valid eFI-CGA score. Missing fields are disregarded in the calculation if the form completion rate is between 80% and 100%, ensuring flexibility while maintaining data integrity for actual frailty assessment for care planning.
Data Storage
To ensure robust data security and patient confidentiality, the eFI-CGA application operates offline, without internet connectivity. Local data storage is implemented, saving data directly onto the hard disk in comma-separated value (CSV) format. CSV files are readily accessible using Microsoft Excel or any text editor available on standard computers. Upon data processing, the application automatically saves all raw data (user inputs for each variable, enabling error checking), calculated data (coded values for algorithm debugging), action-required data (items needing clinical follow-up), and a screenshot of the completed eCGA form (for input verification and comparative assessments) into separate, time-stamped files. To further minimize data loss, the application performs automatic data saves every 3 minutes and upon form closure or reset. This comprehensive data storage strategy ensures data integrity and facilitates future review and analysis of actual frailty assessment for care planning data. Further details on data storage completion are available in the System Input and Output section.
Robustness Handling
To guarantee the software’s reliability and accuracy in automating eFI-CGA scores, rigorous testing was conducted using both simulated and clinical datasets. Manual scoring, based on the established coding scheme and calculation formula, served as the gold standard for comparison against eFI-CGA input and automated eFI score calculations. This comprehensive testing regime ensures the robustness of the actual frailty assessment for care planning tool.
Testing Using Simulation Cases
The software tool underwent initial testing using manually calculated test cases (n = 32) designed systematically for debugging purposes, rather than representing real-world scenarios. These test cases were categorized into four main types to evaluate specific aspects of system performance and accuracy, crucial for ensuring reliable actual frailty assessment for care planning. The first category focused on testing the 80% threshold guard of the eFI-CGA algorithm and its handling of missing data. This included scenarios with 0% form completion (all inputs unanswered) and cases where completed inputs were at, slightly below, or slightly above the 80% threshold. The second category tested the software’s response to entire sections of missing inputs, such as “Mobility” and “Nutrition,” verifying that section-level missing data did not compromise the eFI-CGA calculation, provided the overall form completion rate remained above the threshold. The third category assessed the software’s consistency and flexibility in handling various input values, including minimum, maximum, and intermediate values within the defined range (e.g., 0, 0.5, 1 as per the coding scheme). The final test case category examined the software’s accuracy in capturing and correctly mapping raw input values to coded values between 0 and 1 (Supplementary Materials #3). Additionally, we tested the software’s sensitivity to clinically relevant input changes by systematically manipulating valid input values to observe corresponding changes in the eFI-CGA score, ensuring the tool’s responsiveness in actual frailty assessment for care planning.
Control for Usage Errors
Beyond ensuring accurate score output, we focused on minimizing potential user errors that could affect eFI-CGA computation. Logical input errors were actively prevented. For example, the “Number of Falls” field is automatically set to zero if the user inputs “No” for the “Falls” question, ensuring data consistency and logical coherence. Although the 80% threshold guard allows for up to 20% missing input values, the form is designed to be proactive in guiding users to complete fields. Case-specific alert messages, error messages, and warning messages are displayed to prevent missing fields wherever possible, enhancing data completeness and accuracy for actual frailty assessment for care planning (Figure 3). These proactive measures significantly reduce the likelihood of user-induced errors.
End User Feedback
The application’s features, including the GUI design, input/output methods, and data storage and handling, were iteratively refined based on valuable feedback from clinical users. Clinicians from primary, acute, and geriatric care settings interacted with the application and provided feedback regarding its simplicity and practicality in clinical practice. This user feedback was instrumental in optimizing the software for real-world clinical workflows and ensuring its usability for actual frailty assessment for care planning.
Debugging
Whenever unexpected software behavior or output was observed, or when revisions were made to improve user experience, a thorough debugging process was initiated. The code base of the affected application function and all related program functions were meticulously reviewed. Identified errors or areas for improvement were promptly corrected. Following each correction, the entire suite of test cases was re-applied to rigorously re-test the software, ensuring accurate generation of all expected responses and maintaining the integrity of actual frailty assessment for care planning functionalities. This iterative debugging and testing cycle continued until 100% accuracy in eFI-CGA calculation was achieved across all test and clinical cases.
Test in Clinics
To evaluate the software application’s usability in real-world clinical settings, it was deployed within Fraser Health’s Community Actions and Resources Empower Seniors (CARES) clinical intervention evaluation program [47]. Preliminary data were collected from a small cohort of patient participants (n = 57; mean age = 79.4 ± 6.4 years, female = 45; mean education = 12.5 ± 3.3 years; Clinical Frailty Scale range 3–6) by 14 clinicians across 6 clinics using the eCGA tool. We analyzed the system outputs and the time taken for assessments to assess the practical utility of the software for actual frailty assessment for care planning in a clinical environment.
Results
System Input and Output
The Standalone eFI-CGA application was successfully developed for seamless operation on personal computer devices. Across all 32 test cases, the three-page GUI facilitated user input of essential patient demographics, healthcare professional identification, and all CGA items (Figures 1, 2). Key features including “Preview,” “User Manual,” “Reset,” “Save Records,” and “Calculate eFI-CGA” were fully implemented and functional (Figure 1). Table 1 provides a detailed summary of the purpose and implementation of these main features, all contributing to efficient actual frailty assessment for care planning.
Table 1.
Implementation details of the main features within the standalone eFI-CGA application.
| Main Feature | Purpose | Implementation |
|---|---|---|
| Preview | Provides a one-page screenshot of the entire form for data review and error detection. | When “Preview” is selected, the application captures screenshots of form sections and combines them into a single image for comprehensive review. |
| User manual | Opens the user manual for detailed instructions on eCGA item completion and software usage. | Selecting “User Manual” opens a pre-stored User Manual PDF file from the software’s directory. |
| Calculate eFI-CGA | Automates eFI-CGA score calculation to four decimal places with 100% accuracy. | Upon “Calculate eFI-CGA” selection, the application maps inputs to coded values, applies the FI-CGA algorithm, and displays the resulting eFI score. |
| Save records | Saves user inputs into various file formats for comprehensive data preservation. | The application automatically saves all inputs to multiple files every 3 minutes, upon “Save Records” selection, and when the software interface is closed. |
| Reset | Clears all input fields and returns the user to Page 2 for a new assessment. | Pressing “Reset” clears all fields and navigates the user back to Page 2 of the application. |
| Input validation | Prevents incorrect data entry by enforcing correct input formats and logical relationships. | The application ensures correct input types (e.g., string, integer) and verifies logical consistency between variables during data entry. |
| Missing data handling | Ensures all necessary information is collected for valid eFI-CGA calculation. | When “Calculate eFI-CGA” is selected, the application checks for completion of all required fields and prompts error messages for any missing data to ensure comprehensive actual frailty assessment for care planning data collection. |
The inputs entered into the eCGA form were systematically saved into four distinct data files, ensuring comprehensive data capture for actual frailty assessment for care planning (Figure 4). The software saved raw user inputs (as “TRUE” or “FALSE” for all binary input options), coded values (with “none” indicating missing data), items flagged for clinical follow-up (with “N/A” indicating no action required), and a text summary list of all actions required. Additionally, an image of the eCGA form in its final state was automatically saved upon application closure. Within each spreadsheet, the date and time of assessment were recorded in “date” and “time” columns, facilitating data tracking and comparison across multiple assessments.
Figure 4.
Examples of data files generated and stored by the eFI-CGA application, demonstrating comprehensive data capture for actual frailty assessment for care planning. The top panel shows a spreadsheet containing raw user input data from the eCGA form, with input values represented as “TRUE” or “FALSE.” The middle panel displays a spreadsheet of automatically coded values derived from the input data, using “1” or “0” and “none” to denote missing values. The bottom panel illustrates the action-required spreadsheet, highlighting items flagged for clinical follow-up with “Action Required” or “N/A” for no action needed. Note: Identifiers shown are for illustrative purposes only and do not represent real patient data to protect privacy and confidentiality.
Upon completion of CGA input, the software accurately calculated the eFI-CGA score and presented it with a precision of four decimal places—a value between 0.000 and 1.000 (Figure 5). The calculated score was both displayed on-screen and simultaneously saved within the raw data and calculated data spreadsheets, ensuring comprehensive recording of actual frailty assessment for care planning outcomes.
Figure 5.
Output displays of frailty measurement scores generated by the eFI-CGA application, demonstrating accurate actual frailty assessment for care planning. The top panel shows the automated eFI-CGA score as displayed to the user after pressing the “Calculate eFI-CGA” button upon form completion. The bottom panel illustrates the saved eFI-CGA score within the data spreadsheet, recorded with a precision of four decimal places for detailed analysis and tracking.
Performance
Through rigorous simulation testing and user interaction with the application, we identified and resolved six distinct types of errors (Table 2). These errors included application crashes, functional errors, communication errors, score miscalculations, syntax errors, and incorrect data storage. In each instance of error detection, an informative error message was displayed to the user, as illustrated in examples (Figure 6). Table 2 provides a detailed breakdown of error types, their causes, resolutions, and the phase of software development during which they were identified and corrected, highlighting the iterative refinement process to ensure robust actual frailty assessment for care planning. Systematic identification, diagnosis, and resolution of these errors enabled the release of a software version that achieves 100% accuracy in eFI-CGA calculation, with precision to four decimal places, across all test and clinical cases.
Table 2.
Examples of system debugging and error handling procedures during eFI-CGA application development to ensure accurate actual frailty assessment for care planning.
| Category of Error | Example Description | Cause | Resolution | Stage of Identification |
|---|---|---|---|---|
| Application Crashing | Application terminates unexpectedly when a dropdown list value is not selected. | Default value for dropdown lists was set to NULL. | Changed default dropdown value from NULL to a string type “Select a value” to prevent null reference exceptions. | Initial Version Development |
| Unexpected Function | Form moves out of window frame when user closes form and selects “No” to the “Do you wish to close the application?” prompt. | Software attempted to save a form image even when user declined to close, causing unintended window behavior. | Disabled image saving feature for “No” response to close confirmation to streamline application flow. | Testing Phase |
| Incorrect Communication | Error messages for missing fields persist even after previously missing fields are completed. | Spelling error in the code leading to incorrect variable assignments in error handling logic. | Checked and corrected spelling errors in all error provider objects, ensuring correct variable assignments and preventing interference between error checks. | Testing Phase |
| eFI-CGA Score Miscalculations | eFI-CGA score does not update after input changes are made post initial calculation. | Logic error in code related to formula interpretation and score updating mechanism. | Reviewed and corrected variable mapping, ensuring accurate and dynamic assignment of calculated values upon input changes for reliable actual frailty assessment for care planning. | Testing Phase |
| Syntax Errors | Misspellings and incorrect capitalization in GUI fields, e.g., “Creatinine” spelled as “Creatine.” | Human error during GUI field coding and labeling. | Systematically reviewed and corrected spelling and capitalization errors across all GUI fields to enhance user interface clarity and professional presentation. | Testing Phase |
| Incorrect Data Storage | Data saved under incorrect columns in spreadsheets, leading to misaligned data records. | Incorrect column ordering or discrepancies in the number of columns defined in data storage routines. | Fixed column ordering and added/removed columns as necessary in data saving functions to ensure accurate data alignment and storage in spreadsheets, critical for reliable actual frailty assessment for care planning data management. | Testing Phase |
Figure 6.
Examples of common problems encountered and resolved during the eFI-CGA application development process, ensuring robust actual frailty assessment for care planning. The top panel shows an instance of an application crash that was detected and resolved, which initially forced users to terminate and restart the application. The middle panel illustrates a communication error that was identified and fixed, where users were erroneously prompted to complete fields that were already filled. The bottom panel demonstrates a data misplacement issue that was corrected, where data items were incorrectly saved into shifted columns in the data file. Note: Identifiers are not real to protect privacy.
Table 3.
Efficiency analysis of the main features of the eFI-CGA application, demonstrating optimized performance for actual frailty assessment for care planning.
| Main Features | Time Complexity |
|---|---|
| eFI-CGA Calculation | O(n) |
| Missing Data Handling | O(n) |
| Input Validation | O(1) |
| Reset Functions | O(m) |
| Save Records | O(m + a + 2b) |
Note: “n” represents the total number of possible value choices across all variable and text fields; “m” represents the number of variables and text fields in the eCGA (where m < n); “a” is the number of variables used in frailty calculation; and “b” is the total number of action items.
Efficiency
Time complexity analysis of the Standalone eFI-CGA application revealed efficient performance across its core operations, crucial for practical actual frailty assessment for care planning in busy clinical settings. Table 3 summarizes the time complexities for key operations including eFI-CGA calculation, missing data handling, input validation, reset functionality, and record saving.
The process of transforming raw input values into coded values between 0 and 1 involves visiting each of the multiple value choices for each variable once and storing them in an array. This operation has a time complexity of O(n), where “n” represents the total number of possible value choices across all variable and text fields. Similarly, summing the coded values of m variables and text fields based on the FI formula has a complexity of O(m), where “m” is the number of variable and text fields. When the “Calculate eFI-CGA” button is pressed, the application checks for missing fields and generates specific error messages, which requires revisiting each variable. Therefore, the total time complexity for “Calculate eFI-CGA” remains limited to O(n).
Validating user input values within the eCGA form—checking for logical consistency and appropriate formatting—is performed in constant time, O(1). This is achieved through simple comparisons and matching between the input and pre-defined requirements (Supplementary Materials #2), ensuring rapid input validation for efficient actual frailty assessment for care planning.
The “Reset” operation, which clears all input fields, has a time complexity of O(m), as each of the m input fields needs to be visited once to clear its value.
The “Save Records” operation demonstrates efficient performance with a time complexity of O(m + a + 2b). This operation involves saving data in different formats across multiple spreadsheet and text files. While all variables in the eCGA form are saved once (raw data), and variables used in eFI-CGA calculation are saved once (coded data), action items requiring follow-up are saved twice—once in separate spreadsheets and again in text files. Here, a represents the number of variables used in frailty calculation, and b represents the total number of action items. This optimized saving process ensures comprehensive data preservation without compromising application speed, supporting efficient actual frailty assessment for care planning workflows.
Clinical Test
Initial clinical testing with 57 patients showed that the automated eFI-CGA data from the software matched the manually calculated “ground truth” FI scores to the fourth decimal point (Table 4), achieving a 100% accuracy rate. The patient cohort exhibited a mean eFI-CGA score of 0.2516 ± 0.1123. The average number of comorbidities was 7.9 ± 3.7, and an average of 0.73 ± 1.36 action fields were identified per patient for clinical follow-up, demonstrating the tool’s utility in identifying areas for care planning. The average time taken to automate an eFI-CGA score by completing a full eCGA assessment was 44.5 ± 12.8 minutes. However, when using pre-recorded CGA items, the time to automate the eFI-CGA score significantly reduced to 8.6 ± 3.3 minutes. This indicates the software’s feasible use in clinical applications, particularly in scenarios where CGA data might be pre-existing or when assessments are conducted more rapidly, enhancing actual frailty assessment for care planning efficiency.
Table 4.
Results from initial clinical tests of the standalone eFI-CGA software application in medical clinics, demonstrating its practical utility for actual frailty assessment for care planning.
| Patients (n = 57) 45 females; 12 males | Mean | S.D. | Minimum | Maximum |
|---|---|---|---|---|
| Age (year) | 79.4 | 6.4 | 65 | 97 |
| Education (year) | 12.5 | 3.3 | 4 | 22 |
| Clinical Frailty Scale | 3.69 | 0.87 | 3 | 6 |
| Comorbidities | 7.9 | 3.7 | 1 | 18 |
| Number of actions needed | 0.73 | 1.36 | 0 | 6 |
| eFI-CGA | 0.2516 | 0.1123 | 0.0429 | 0.5783 |
| Time for assessment (minute) | 44.5 | 12.8 | 20 | 66 |
| Time from item entry (minute) | 8.6 | 3.3 | 5 | 24 |
Discussion
In this study, we successfully developed an eFI-CGA software application designed to support frontline frailty assessment and enhance actual frailty assessment for care planning. This software tool effectively automates data recording, storage, processing, and calculation of the eFI-CGA score from user inputs within an electronic CGA form interface. The software provides a reliable eFI score along with comprehensive data files including an image of the completed eCGA form and identified areas requiring follow-up for care planning. Implemented on Microsoft’s WinForms platform, the application is optimized for convenient use on personal computers. To ensure user-friendliness and familiarity, the software’s graphical user interface mirrors the design of a well-established paper-based CGA form. Rigorous testing and debugging, utilizing systematically designed test data, real-world data, initial clinical data, and user feedback, confirmed that the eFI-CGA calculation achieves 100% accuracy with precision to four decimal places. The Standalone eFI-CGA software enables multiple saves of user input at various data entry and processing stages, facilitating future reference and review of actual frailty assessment for care planning data. Clinical feedback indicates that the eFI-CGA software is efficient and well-suited for implementation in clinical settings.
Frailty poses a significant challenge to effective healthcare delivery for older adults, particularly those with complex health conditions and care needs. Despite evidence indicating that frailty can be improved with early identification and management [13], there remains a shortage of clinically feasible and valid frailty assessment tools to effectively guide care planning. Current electronic frailty measurement tools that generate FI scores are often based on ad hoc assessments rather than the more comprehensive CGA, which has proven effective in clinical trials. While recent research leveraging electronic health records to calculate eFI has accelerated frailty screening [35–44], current efforts are focused on validating these tools for individual patient decision-making. In geriatric medicine, CGA remains a cornerstone assessment, frequently conducted using paper forms or with scribes. Our technological contribution builds upon a deep understanding of frontline user needs, informed by the developer of the original paper-based CGA form from the 1990s [45].
The FI-CGA is not yet a standard component of primary care. Tools like the UK eFI [35], though derived from general physician records, are often viewed as screening tools requiring subsequent verification assessments for individuals screening positive. Our motivation stems from a broader initiative to integrate geriatric principles into routine care, aiming to shift early frailty management towards enhanced primary care. Achieving this requires appropriate and accessible tools. Therefore, the development of our software tool aims to transition the proven effectiveness of paper-based CGA, traditionally used by geriatricians, into the EMR environment as an eFI-CGA. Our research has successfully delivered the eCGA and eFI-CGA software as automated geriatric and frailty measurement tools, empowering healthcare providers to effectively manage frailty at the point of care and improve actual frailty assessment for care planning. Even with the availability of an EMR-integrated version, the use of our software by primary and acute care physicians facilitates information integration across healthcare settings, informing comprehensive and individualized care plans.
The Standalone eFI-CGA software application addresses several critical needs in frailty research. It provides a platform to compare outputs from different eFI-CGA implementations used by various EMR vendors, which often lack standardization and involve function/system trade-offs inherent in EMR implementation. Currently, different EMR systems automate eFI-CGA using varied approaches, complicating the merging of measurement outputs. Future analyses will compare EMR-embedded versions with our Standalone eFI-CGA to ensure implementation feasibility, transparency, and accurate, robust eFI-CGA score outputs, enhancing actual frailty assessment for care planning across systems. Furthermore, the eFI-CGA software tool enhances data collection efficiency for both eCGA and eFI-CGA. Healthcare professionals can initiate data collection using any Windows-based desktop or laptop in diverse settings, including patient homes, without EMR access limitations.
Integrating eFI-CGA into clinical programs can significantly benefit the effective evaluation of intervention outcomes. In the Community Actions and Resources Empower Seniors (CARES) program [47], healthcare professionals utilize eFI-CGA results to develop personalized treatment plans and referrals to community-based programs [48, 49], directly leveraging actual frailty assessment for care planning. The eFI-CGA is measured at multiple time points to track changes in frailty as a CARES intervention outcome. Once validated for widespread clinical use, the eFI-CGA has the potential to become standard practice in integrated healthcare for older adults, including primary care and home medicine, promoting early frailty identification, informed decision-making, and improved care planning. Specifically, the eCGA form, with its automated FI-CGA output, enables healthcare professionals to tailor interventions to individual patient needs more effectively. The Standalone eFI-CGA software tool’s compatibility with any personal computer facilitates its adoption across various healthcare professionals and settings, streamlining actual frailty assessment for care planning processes.
Several technical limitations warrant consideration for future software development and implementation efforts. The application’s efficiency could be further optimized in future releases using more advanced implementation techniques. For example, the “Calculate eFI-CGA” function could integrate missing input checks within the variable mapping step itself. However, given the relatively small number of variables in the eCGA form, such efficiency improvements might not significantly impact user experience. The eFI-CGA score is currently generated rapidly—within milliseconds—on most devices. Time complexity for other features like saving records or resetting is also efficiently O(n) and unlikely to show substantial gains from further optimization.
Current operating systems prohibit simultaneous reading and writing to the same memory location, which can trigger error messages in the software program. Consequently, users cannot open saved data files while the software is running due to its automatic background data saving process (every 3 minutes). To mitigate this, data files are hidden from user access during active software use and become accessible once data entry is complete and the program is closed. A refined approach for future implementation could involve storing data separately on a server or in the AppData folder, which is hidden by default and designed for application settings and data. However, this would require the program to run from the host computer rather than a portable memory stick, as currently designed.
The choice of WinForms as the development platform also presents inherent limitations in GUI design. The eCGA form’s appearance may vary slightly or appear somewhat blurry depending on the user’s device resolution settings. Enhancements such as zoom in/out functionality could improve user experience but might be more readily implemented with newer WinForms versions. As healthcare technology advances and newer Microsoft Windows versions become universally accessible, platforms like WPF libraries will offer greater GUI design flexibility.
Importantly, for future research and development, establishing the psychometric properties of the eFI-CGA is crucial before its implementation into standard healthcare practice for actual frailty assessment for care planning. The standalone software tool facilitates eFI-CGA score testing across diverse healthcare settings and by various professionals, making it valuable for reliability and validity analyses. Our ongoing research focuses on completing longitudinal clinical assessment data collection to assess the reliability and validity of the eFI-CGA tool and its clinical applications. In related work, we are developing and testing an EMR-integrated eFI-CGA, incorporating insights from the current software development to ensure transparency and feasibility for widespread clinical adoption in actual frailty assessment for care planning.
Restrictions for Use
The Standalone eFI-CGA application was developed as part of the research project “Frailty assessment for older adults at point of care: Validating the electronic Comprehensive Geriatric Assessment / Frailty Index (eFI-CGA).” This development was a collaborative effort between Fraser Health Authority (FHA) Community Actions and Resources Empowering Seniors (CARES) program and Nova Scotia Health Authority (NSHA) Geriatric Medicine Research Unit (GMRU). All rights are reserved by the Stand-Alone eFI-CGA Project Team (CIHR Grant # CIHR-PJT-156210 2018-2022). The Standalone eFI-CGA software is not available for sale or commercial distribution. The released software is provided for research, academic, and clinical purposes only, with prior written permission from the project’s primary investigator (XS, corresponding author) or designated representative. Parties interested in using the software should contact the primary investigator ([email protected]) to obtain a Standalone eFI-CGA Research Agreement Form. Upon signing the agreement, a link to the encrypted application will be provided. Any publications or media communications utilizing the eCGA / eFI-CGA software must include the following citation: “The eFI-CGA Project Team (CIHR Grant CIHR-PJT-156210) 2018-2022: Frailty assessment for older adults at points-of-care: Validating the electronic Comprehensive Geriatric Assessment / Frailty Index (eFI-CGA).”
Readers are invited to request access to the software for research purposes via the link below.
Please contact the corresponding author, [email protected], to obtain the password required to download the software from the provided link.
Data Availability Statement
All datasets generated during this study are included within the article and its Supplementary Material.
Ethics Statement
This study received ethical approval from the Fraser Health Human Research Ethics Board (FHREB2018-080). All patient and clinician participants provided written informed consent to participate in this research.
Author Contributions
KS: Application coding and drafting technical manuscript aspects. MB, BC, MZ, BT: Testing, debugging, data processing/analysis, manuscript preparation support. GP, AG, RM: Critical input on user requirements, user manual, clinical data collection, manuscript editing. KR: Originated paper-based CGA form, allowed form use as software template, critical input on user requirements/guidance, clinical data collection, manuscript editing. XS: Conceptualized and supervised eFI-CGA software development, obtained research funding, supervised data processing/analysis, co-prepared initial manuscript draft. All authors: Reviewed and approved final manuscript version.
Conflict of Interest
The authors declare that no commercial or financial relationships exist that could be perceived as a potential conflict of interest.
Acknowledgments
The authors express gratitude to Drs. Alison Rodger, Leah Nemiroff, Olga Theou, and Barry Clarke for critical input on usage requirements and user guide development; Kate Keetch and Sherri Fay for research project management and administrative support. Sincere acknowledgments to Harveer Sihota, Sarah Crowe, Maria Judd, Laura Housden, Sabrina Wong, Drs. Roger Wong, Andrew Sixsmith, John Muscedere, Brenda Hefford, Hui Guo, and Victoria Lee for collaborations and project support. The authors thank Hilary Low and Tara Arvan for assistance with data processing/analysis and manuscript revision. Sincere acknowledgments also extend to Surrey Memorial Hospital, Fraser Health Authority, Nova Scotia Health Authority, Dalhousie University Geriatric Medicine Research Unit, Simon Fraser University, Doctors of BC, and the Canadian Frailty Network for their support.
Footnotes
Funding. This research was supported by an operating grant from the Canadian Institutes of Health Research (CIHR; Grant # CIHR-PJT-156210) and trainee support from Surrey Hospital Foundation.
Supplementary Material
The Supplementary Material for this article is accessible online at: https://www.frontiersin.org/articles/10.3389/fpubh.2020.00089/full#supplementary-material
Click here for additional data file. (153.6KB, pdf)
Click here for additional data file. (23.7KB, xlsx)
Click here for additional data file. (247.7KB, jpg)
References
[References]
Associated Data
Supplementary Materials
Click here for additional data file. (153.6KB, pdf)
Click here for additional data file. (23.7KB, xlsx)
Click here for additional data file. (247.7KB, jpg)
Data Availability Statement
All datasets generated during this study are included within the article and its Supplementary Material.
