Knowledge Gaps
Knowledge Gaps analyzes successful information-seeking turns. It calculates a 0–100% heuristic confidence from whether knowledge search ran, results/relevance, uncertainty language, and response length.
The gap score is a triage heuristic, not an AI certainty meter. For successful information-seeking turns, SmartSite starts at full confidence and subtracts weight for missing search, no results, weak vector relevance, uncertainty wording, or a suspiciously short answer. The resulting queue points administrators toward conversations that deserve source and retrieval review.
Work through this page by starting when you choose a date range and threshold: 50, 60, 70, 80, or 90 percent; default is 70. Finish by making sure you synchronize and retest in a new conversation; if that check fails, the field descriptions help identify whether the problem belongs to content, behavior, transport, or operations.
What this feature does and when to use it
Section titled “What this feature does and when to use it”Use it as a triage queue for source improvements, not as a factual correctness score.
Use this feature in the following situations:
- You want a prioritized list of questions that may not be adequately covered by current knowledge.
- You need to inspect why a low-confidence item was scored and which files were searched.
- You want to measure whether a source/metadata improvement changes the repeated question.
Where to find it
Section titled “Where to find it”Before you begin
Section titled “Before you begin”- SmartSite Assistant is installed and activated.
- You are signed in with an account that can manage WordPress options.
Set it up step by step
Section titled “Set it up step by step”- Choose a date range and threshold: 50, 60, 70, 80, or 90 percent; default is 70.
- Review analyzed count, gap count/rate, average confidence, trend, and recurring topics.
- Open a low-confidence item.
- Read its reason signals, response, and searched files/scores.
- Classify the issue as missing source, weak retrieval, unclear question, instruction problem, or acceptable uncertainty.
- Improve the smallest relevant source/metadata/instruction.
- Synchronize and retest in a new conversation.
Fields, controls, and important values
Section titled “Fields, controls, and important values”Knowledge-gap signals are a prioritization aid for finding questions that may not have received strong supporting information. Low confidence can point to missing pages, weak metadata, or wording that did not trigger retrieval. It is not a factual score for the answer itself, so read the conversation and retrieval evidence before changing content or assuming the AI was wrong.
| Field, control, or status | What SmartSite Assistant does with it | How to use it and why it matters |
|---|---|---|
| Confidence baseline | Starts at 1.0 and subtracts signals; clamped to 0–1. | Choose “Confidence baseline” from observed needs rather than guesswork. Boundary values can affect availability, detection, display, or reporting; test a normal case and an edge case so useful conversations are not sacrificed for an arbitrary number. |
| Major deductions | No knowledge −0.4; top vector score <0.30 −0.4; queries with no results −0.35; uncertainty −0.25. | These signals make confidence fall sharply when knowledge was absent, retrieval was very weak, or the response expressed uncertainty. They are useful for prioritizing review, but each conversation still needs human reading before a page is added or rewritten. |
| Other deductions | Top score <0.50 −0.25, <0.70 −0.1; search not triggered −0.15; short answer pattern −0.1; legacy search without vector −0.1. | Smaller deductions capture weaker retrieval, missing search activation, short-answer patterns, and legacy evidence. Several can combine into a low score; use the breakdown to choose a likely investigation path rather than assuming every deduction represents an incorrect answer. |
| Recurring topics | Simple word frequency using an English stop-word list; interpret cautiously for other languages. | Use “Recurring topics” to isolate the activity relevant to one question. The result can reveal where an answer failed, but the filter itself does not fix anything; improvement comes from the configuration change made after reviewing the evidence. |
| Pagination | 15 items per page. | Interpret “Pagination” with the unit, range, and default shown beside it. Change one boundary at a time and test just below and above it; tighter numbers may protect cost or safety but can also remove legitimate context or activity. |
How to confirm it is working
Section titled “How to confirm it is working”Confirm Knowledge Gaps with the smallest representative end-to-end test. When the result differs from the success description, keep the exact input and state so troubleshooting can identify the responsible layer without changing unrelated AI settings.
Practical example
Section titled “Practical example”A 45% item with “queries but no results” suggests a coverage/indexing problem; a low score with a correct cautious answer may need no change.
Recommended practice
Section titled “Recommended practice”- Change one part of Knowledge Gaps at a time and keep a short record of the previous value and test result.
- Turn one observed pattern into one controlled configuration change, then compare new conversations with the earlier evidence.
Important warnings
Section titled “Important warnings”Common problems and focused checks
Section titled “Common problems and focused checks”| Problem | What to check and what to do next |
|---|---|
| Knowledge Gaps is missing or does not match this guide. | Confirm the plugin is active and the account can manage WordPress options. Narrow the date and filter scope, then open source records instead of relying on an empty chart or summary alone. |
| A change on Knowledge Gaps does not produce the expected result. | Keep the exact notice and test case, then review the browser console and WordPress/PHP log. Narrow the date and filter scope, then open source records instead of relying on an empty chart or summary alone. |
Screen reference
Section titled “Screen reference”- Capture
- Show Knowledge Gaps at 70% with sanitized summary cards, recurring topics, and one expanded item showing signals and file scores.
- Show
- Date/threshold, analyzed/gap/rate/confidence cards, trend, topics, detail evidence
- Viewport
- Desktop, 1440 × 900
- Annotate
- Use numbered callouts only for controls referenced in the procedure.
- Redact
- OpenAI keys, tokens, secrets, personal information, private URLs, IP addresses, and conversation text