Evaluation guide

Before You Trust a New Aviation App: A Pilot’s Ground-Test Checklist

Start with the job you want the app to do, then record what you can actually verify. A polished demonstration or an AI-built label does not establish reliability.

By Nick Rae · Published 2026-10-24

How this guide was prepared

Editorial checklist prepared October 24, 2026 from the FAA’s active AC 91-78A, Aviation Weather Center API documentation and Apple privacy documentation. These sources establish their own guidance and disclosure boundaries; the evaluation steps below are our recommendations. No vendor account, paid feature, cockpit workflow or AI answer quality was tested for this guide. Examples are hypothetical ground exercises. This is not flight instruction, a safety rating or authorization to replace required equipment or information.

Choose the job before the app

Write one sentence describing the task you want to perform. “Review a practice answer after a study session” and “calculate information I intend to use for a flight” need different scrutiny. An app can be useful for one narrow task while leaving another unsupported.

Identify what happens if the result is wrong, old, missing or unavailable. Start evaluation on the ground with a demonstration, documented sample or synthetic record. Keep experiments separate from a genuine logbook, live flight plan or operational decision. A list of completed checks does not establish that an app is suitable for flight use.

Label the evidence behind each claim

Use the same standard for an established product and a new app, whether its code was written with AI assistance or by another process. The development label alone tells you little about a particular function. Record the claim, its source and what remains unknown.

Three useful evidence labels — none is a safety certification
Label	What it establishes	What it does not establish
Vendor claim	What the provider advertises or documents.	That the feature works correctly in your circumstances.
Public-source check	A dated review of a specific published detail.	A hands-on product test or independent validation.
Observed task test	A recorded result for stated inputs, device and version.	Reliability outside that tested task or after an update.

Trace the data and its age

Look for a named upstream provider, coverage, relevant document edition and a visible timestamp or validity period. Separate the time an observation was made from the time the app fetched it. A recent screen refresh does not necessarily mean a new report.

The Aviation Weather Center documents METAR observations and TAF forecasts as different products. A valid API request can return no available data. In your evaluation, check whether the app makes missing or stale information clear instead of presenting a normal-looking result. Do not assume every product follows one update schedule.

Sources: NOAA/NWS Aviation Weather Center — Data API products and response meanings.

Ground-test the boundary cases

Use provider-supported demonstrations or harmless synthetic inputs. Predict the expected behavior before you try a case. Do not interfere with someone else’s service, submit operational messages or manufacture records in a real account just to provoke an error.

Units and dates: inspect the displayed units, time zone and date around midnight. If the app accepts different units, confirm that labels and saved records remain consistent when you change them.
Missing information: leave an optional field empty or use a documented no-data example. An unknown value should remain distinguishable from a measured zero or a favorable result.
Unsupported cases: check the documented region, aircraft, device and feature limits. Look for a clear explanation when the requested case is outside those limits.
Connection loss: on the ground, inspect what remains available without a connection. Distinguish saved content from information that needs a successful refresh, and check how the last update is shown.
Interruption: background the app, rotate or resize the screen, then return. Check whether the input, selected aircraft or station, units and unfinished work remain understandable.

Compare one result with an appropriate reference

Choose a task whose expected result you can establish independently. Confirm that the reference applies to the same aircraft or subject, configuration, units and effective date. Two apps displaying the same answer may share an upstream source; agreement alone is not an independent check.

For AI study feedback, open the cited publication yourself. Find the actual section and examine the context and exceptions. Record a missing citation, unsupported conclusion or unresolved disagreement rather than accepting confident wording. Take instructional questions to your instructor. This guide supplies no performance calculation or flight decision.

Check privacy before uploading information

Read the privacy policy and relevant account terms before importing a logbook, flight track, voice recording or document. Ask what leaves the device, which services receive it, how long it is retained, whether it is used to train models, and how deletion is requested. Treat an unanswered question as unknown.

Apple says App Store privacy information is self-reported by developers, and its developer guidance includes third-party partners’ practices. Compare the label with the policy and permission requests; it is not an independent security audit. Begin with synthetic material and only the permissions needed for the task.

Sources: Apple Support — App Store privacy information and developer reporting; Apple Developer — app privacy details and third-party practices.

Test export, recovery and support

For a record-keeping app, export a small synthetic dataset and open it outside the app. Inspect dates, units, notes and any attachments you expect to retain. Readable export, successful re-import and a recoverable backup are separate questions. Do not delete or replace genuine records to test recovery.

Check what happens if the subscription ends, an account cannot sign in or the provider closes. Identify which information you can retain independently. Find a support contact and release history, and ask the developer about a specific unresolved behavior. A prompt answer is useful evidence about support, not proof of software correctness.

Keep a small evidence log

A brief record is more useful than a vague “worked for me.” Keep screenshots free of personal information. These example entries are a worksheet, not reported results from a product test.

Ground-test evidence log template
Record	Write down
Environment	Date, app version, device, operating system and account tier.
Task and inputs	One task, synthetic input values and relevant settings.
Reference	Source, edition or timestamp, and your expected result.
Observation	Actual result, message or screenshot; mark not tested where appropriate.
Limit and next action	Unresolved difference, supported scope and the question to investigate.

Make a bounded decision, then retest after change

Keep separate outcomes: suitable for further ground exploration, unsuitable for the intended task, or insufficient evidence. A result limited to study or post-flight record review does not transfer to navigation, performance planning or another operational function.

Revisit the relevant cases after changes to the app, operating system, data provider, device, subscription or permissions. Record the new version rather than carrying forward an old test result without qualification. If an important result disagrees with its reference or a failure is unclear, investigate before relying on that function.

Keep the guidance in scope

The FAA lists AC 91-78A as active. It addresses Part 91 operators replacing required paper information or using database and software applications as part of EFB functionality. It is advisory, not a regulation; its stated method has conditions. Its implementation guidance includes interface assessment and cross-checking entered and computed information. Read the full circular for its applicability, rather than treating this general app checklist as an EFB compliance procedure.

John Zimmerman’s October 19, 2026 iPad Pilot News article prompted this guide’s practical question: how can a pilot examine a new app’s evidence and limitations before depending on it? The checklist here is our own editorial framework, not a product ranking or a reported test of the directory’s listings.

Sources: FAA AC 91-78A — active document status and scope; FAA AC 91-78A — full circular, especially sections 9 and 11; iPad Pilot News — John Zimmerman, October 19, 2026; editorial background.

Sources

Browse related categories

Nick sells aviation study workbooks that compete with some listed study tools. Read the ownership and research disclosure. Use current official publications and instructor guidance before relying on an aviation tool.