As you decide whether to use an app, you can use the following criteria (ABACUS), along with testing according to this framework, to help you evaluate it:
- Is the medical information contained in the app based on sound medical research and evidence? Can the information in the app be verified by another source?
- Are there references/sources included so that you can verify the information? Are these references reliable? (For example, a citation to a drug company website does not have the same weight as an article from JAMA.)
- Are there grammatical and spelling errors? (This may be a "tell" - if the information isn't even spelled correctly, maybe the information itself isn't correct.)
- Does the app do what it intends to do? Is there any potential for patient harm?
- Is the information showing just one point of view or is it sponsored by a company that is trying to sell something?
- What kind of organization sponsored the app? A pharmaceutical company? A non-profit organization? A reputable journal?
- Is advertising clearly marked and distinguishable from the informational/medical content? Can you tell if the information you are reading is advertisement?
- Does the app use data improperly to promote a position or a product, or is it unbiased/neutral?
- Who developed the app? What are the person's or sponsoring organization's credentials? Are they an expert in the content presented in the app? What do you know about them?
- Is the person backed by a known organization? (Be careful here... some "organizations" may simply be unreliable groups operating out of someone's basement; try to go with auhoritative sources, like the National Library of Medicine.)
- Do experts review the content provided in the app and are these "experts" real authorities on the content?
- Can you easily find contact information in the app or on its download/information page? Check the about us link/seller information, usually found on the app's download page. What is the purpose of the organization? Is it trying to sell something or is it an unbiased, peer-reviewed information source?
- When was the app created and/or last updated?
- Does the app provide regular updates when new content or technological upgrades are required?
- Has there been more recent research on the content in the app? Many medical treatments change with the publication of new studies. What was published a year ago may be outdated now.
- Does the app work reliably and stably on the device you are using?
- Is navigation smooth and intuitive?
- Is the app efficient and effective? For example, is the type of content usable on a small screen (e.g., radiological images)? Is data entry easy?
- Is the app appropriate for the target audience (e.g., patient info apps are in plain language)?
- Does the app author provide technical support for the app?
- Is the app stand-alone (meaning you can use it without a wi-fi or Internet connection)? This is just a good thing to note so you are aware about whether the app can be used without an Internet connection.
- Is the medical information presented in the app complete?
- Are there sources given for additional information?
- Who is the target audience - is the app targeted for use by medical professionals, patients, others?
Be Sure to Ask Yourself:
- Why did the person/organization create the app?
- What's in it for them or are they trying to sell me something?
- Is the creator of the app an expert in the content presented in the app?
- Can I verify the information being presented to me in the app and is it accurate?
- Is there a way I can contact the app developer to provide feedback or ask a question?
- Are there any login requirements or privacy issues that I need to know about if I choose to use this app? Will my use of this app be tracked in any way?
- Is there a disclaimer that states any impact on clinical decision making, patient safety?
TEST before you use:
- TEST the app before you use it in clinical care - create clinical scenarios and test.
- As you test, observe and evaluate the app according to the above ABACUS framework. Does it pass the Accurate, un-Biased, Authoritative, Current, Usable, Scope/Completeness benchmarks in multiple case scenarios?