Understanding the Bayesian Analysis Framework

Bayesian reasoning is the mathematically principled way to update beliefs in light of new information. You start with a prior — what you believe before seeing the evidence — and you revise it into a posterior — what you should believe after seeing the evidence.

This application performs that revision iteratively: each evidence item is applied one after another, each multiplying the running probability by a factor derived from its Likelihood Ratio and your stated Confidence in it. The result is a transparent, auditable chain of reasoning.

The engine converts the prior probability into odds, multiplies by each evidence item's effective Likelihood Ratio, then converts back to a probability. In compact form:

The step-by-step logic:

1. Convert your Prior Probability P into odds: Odds = P / (1 − P)
2. For each evidence item i, compute its Effective LR: LR_eff = LRᵢ ^ confidenceᵢ
3. Multiply the running odds by each Effective LR in sequence.
4. Convert the final odds back to a probability: P_posterior = Odds / (1 + Odds)

The Hypothesis is the central claim you want to evaluate. It must be a clearly defined, falsifiable statement — something that can be either supported or contradicted by real-world observations.

A well-formed hypothesis is specific enough that you can meaningfully ask: "Does this piece of evidence make this claim more or less plausible?" All evidence you enter into the system is evaluated against this single hypothesis.

The Prior Probability is your starting belief in the hypothesis before considering any current evidence. It represents the background plausibility of the claim based on existing knowledge, prior studies, expert consensus, or institutional experience.

• 0.50 — You have no initial lean; the hypothesis is as likely as not. A reasonable default when you lack prior information.
• 0.30 — You consider the hypothesis unlikely based on prior knowledge.
• 0.70 — Prior experience or literature already suggests the hypothesis is plausible.
• 0.10 – 0.20 — Strong reason to doubt; use only if existing evidence strongly points away from the hypothesis.
• 0.80 – 0.90 — Substantial prior evidence already exists in favor; evidence is being used to confirm a near-established view.

In Bayesian logic, if you set a Prior to exactly 0 or 1, no amount of evidence can ever change your mind. Multiplying by zero always results in zero.

The Decision Threshold is the posterior probability value at which you consider the hypothesis sufficiently supported to justify a decision, intervention, or recommendation. It separates exploratory uncertainty from actionable confidence.

• 0.60 — Low bar; appropriate for exploratory analysis or flagging issues for further investigation.
• 0.70 – 0.75 — Moderate confidence; suitable for programmatic recommendations or preliminary policy decisions.
• 0.80 – 0.85 — High bar; appropriate for significant resource allocation, formal reports, or public-facing recommendations.
• 0.90+ — Very high bar; use only where the cost of being wrong is severe.

Each evidence item represents a discrete real-world observation that bears on the hypothesis. Evidence can come from a wide variety of sources:

• Field observations and site visit reports
• Survey data and community interviews
• Statistical indicators and administrative data
• Expert assessments and peer-reviewed findings
• Policy documents and program outcomes

Evidence Direction

Each item must be labeled as either supporting (For) or contradicting (Against) the hypothesis. This directionality is built into the Likelihood Ratio: an LR above 1.0 supports the hypothesis, and an LR below 1.0 contradicts it. Providing both types of evidence produces a more balanced and credible analysis.

The Likelihood Ratio quantifies how much a specific piece of evidence shifts the probability of the hypothesis. Formally, it is the ratio of two probabilities:

In plain language: How much more (or less) likely is it that we would observe this evidence if the hypothesis is true, compared to if it were false?

If you are unsure what Likelihood Ratio value to assign, you can use the built-in Preset dropdown. The scale translates qualitative evidence strength into approximate LR values commonly used in Bayesian reasoning and intelligence analysis.

These values are guidelines only. Users may still manually type custom Likelihood Ratios if more precise estimates are available.

The Core Distinction

Every piece of evidence carries two independent questions:

1. If this evidence is real and accurate, how much does it update my belief? → This is the Likelihood Ratio.
2. How much do I trust that this evidence is real and accurate in the first place? → This is your Confidence.

You might have an observation that, if true, would be very powerful evidence — but it comes from a single, unverified source. You give it a high LR because the finding is significant, but a low Confidence because you are uncertain about its quality. The model gracefully handles both dimensions simultaneously.

The Mathematical Effect

Confidence works as an exponent applied to the Likelihood Ratio:

What this means in practice:

• At Confidence = 1.0, the evidence is applied at full strength: LR_eff = LR
• At Confidence = 0.5, the evidence is applied at half-power: LR_eff = √LR
• At Confidence = 0.0, the evidence has no effect: LR_eff = 1.0 (neutral)

Crucially, the direction of the evidence is preserved — reducing confidence simply pulls the effect toward neutrality, it does not reverse it.

Confidence Reference Scale

What Degrades Confidence?

• Source quality: Unverified reports, single informants, rumors → lower confidence
• Sample size: One village versus a survey of one hundred villages → lower confidence for small samples
• Observation recency: Data from several years ago may no longer reflect current conditions
• Measurement method: Subjective assessments are less reliable than objective measurements
• Potential bias: Evidence gathered by parties with a stake in the outcome
• Consistency: Evidence that contradicts many other reliable sources

This indicator shows the percentage-point change in the posterior probability attributable to a single evidence item. It answers the question: "Of all the evidence I entered, which one moved the needle the most?"

Use this to:

• Identify the most influential factors driving your conclusion
• Spot evidence that may be over-weighted (very high impact with low confidence)
• Prioritize which data to verify or expand in further fieldwork
• Communicate to stakeholders which findings are most critical

The notes field is optional but strongly recommended for any analysis that will be reviewed, shared, or revisited. Document:

• The source of the evidence (report name, survey date, institution)
• How the observation was collected (methodology)
• Why you assigned the specific LR and Confidence values you chose
• Any caveats or limitations associated with the evidence
• Links to supporting documents or data tables

This documentation is what separates a transparent, auditable analysis from an opaque black box. It allows others — and your future self — to critically evaluate the reasoning behind each probability update.

The core limitation of Naive Bayes is that it treats every evidence item as independent of the others. In reality, two observations may be causally linked: Evidence A may cause Evidence B. If you enter both independently, you apply the same underlying signal twice and double-count it.

The Fieldwork Solution: Composite Evidence

When two observations are causally connected, merge them into a single composite evidence statement and assign one representative Likelihood Ratio to the combined observation:

This single entry captures the full meaning without double-counting. The Likelihood Ratio you assign should reflect the combined strength of both signals, and your Confidence should reflect how well-documented the causal link is.

Full Bayesian Networks (also called Bayesian Belief Networks) represent causal dependencies between variables using a Directed Acyclic Graph (DAG). Each variable can conditionally depend on others, and inference propagates through the network via algorithms such as:

• Belief Propagation (Message Passing) — efficient on tree-structured networks
• Variable Elimination — exact inference by marginalizing over hidden variables
• MCMC Sampling — approximate inference for large, complex networks

• Fully Offline Operation: The entire computation runs inside your browser. No internet connection, server, or external library is required — critical for remote or low-connectivity field environments.
• Low Cognitive Overhead: Analysts need only two numbers per evidence item — a Likelihood Ratio and a Confidence value — rather than full conditional probability tables with potentially dozens of entries.
• Empirically Robust: Despite the independence assumption, Naive Bayes classifiers consistently perform well in practice — even when variables are correlated. The model is more resilient to mild dependency violations than its theoretical limitations might suggest.
• Transparent Reasoning Chain: Every evidence item's contribution to the posterior is visible and individually interpretable. The analysis can be fully explained to policymakers, community leaders, or auditors without specialist training.
• Auditable and Reproducible: All inputs — prior, LRs, confidence values — are explicit and documented. A colleague can reproduce, challenge, or refine the analysis by adjusting any single parameter.
• Graceful Handling of Uncertainty: Through the Confidence parameter, the model explicitly represents the varying quality of real-world field evidence — something many simpler frameworks cannot do.

Human analysts are naturally prone to Confirmation Bias—the tendency to only collect evidence that supports our existing worldview. This application monitors the composition of your evidence list in real-time.

Triggering a Warning

• Homogeneous Evidence: If 100% of your items have an LR > 1.0, the system will flag a "Blindspot Warning." It is statistically rare in complex governance environments for no contradictory data to exist.
• Cluster Over-reliance: If 80% of the movement in your posterior probability comes from a single cluster (e.g., "Economic Data"), the system warns of a lack of multidimensionality.

Moving a slider based on "gut feeling" can be hard to defend in a formal audit. The Confidence Matrix Calculator (🧮) uses a 3-point triangulation to standardize how evidence quality is scored:

By averaging these three scores, you transform a subjective "feeling" into a methodological assessment that can be documented in your final report.

Struggling to pick an LR? Use the "Contrast Question":

If you would expect to see this evidence 60% of the time if the claim is true, but only 20% of the time if it's false, your LR is 3.0 (60/20).

If you are new to Bayesian reasoning, do not worry about the formulas at first. Start with these three simple steps and focus on the logic of the evidence.

1. Set the Prior to 0.5: This represents an initial position of uncertainty — you are essentially saying “I have no strong belief yet.” In Bayesian analysis, this is often called a non-informative starting point. From there, the evidence will update the probability.
2. Use the Likelihood Slider Relatively: Think about how likely the evidence would appear if the hypothesis were true.
   • If the evidence would very likely appear when the hypothesis is true, move the slider to the right (supporting evidence).
   • If the evidence would rarely appear when the hypothesis is true, move the slider to the left (evidence against the hypothesis).
   The goal is not perfect numbers, but a reasonable estimate of how the evidence changes the likelihood of the hypothesis.
3. Use the Confidence Matrix: If you are unsure about the reliability of the information, click the 🧮 icon. The matrix helps estimate how trustworthy or strong the evidence source is. This allows the system to adjust the influence of that evidence automatically.

Bayesian thinking is closer to how a detective works than how a judge works. A judge makes a final decision at the end of a trial, but a detective constantly updates their belief as new clues appear.

When new evidence appears tomorrow, the probability should change. Updating beliefs in light of new data is not inconsistency — it is the core principle of rational scientific reasoning.

• Using Extreme Likelihood Ratios Too Quickly: Assigning very large Likelihood Ratios (for example LR = 20) for evidence that is based only on rumors or weak observations. In most practical analyses, moderate values (around 1.5 – 3.0) are safer unless the evidence is extremely strong or experimentally verified.
• Ignoring Contradictory Evidence: Adding many pieces of supporting evidence while intentionally ignoring information that contradicts the hypothesis. This is a classic confirmation bias. A robust analysis should always consider both supporting and opposing evidence.
• Confusing Evidence Strength with Source Reliability: Remember that these two concepts are different:
  • Likelihood Ratio (LR): how strongly the evidence supports or contradicts the hypothesis.
  • Confidence: how much you trust the source or quality of that evidence.
  Strong evidence from an unreliable source should still be treated cautiously.

The decision threshold defines the minimum probability required before taking action on a hypothesis.

In Bayesian analysis, probability alone does not automatically determine a decision. A threshold acts as a boundary between "continue investigating" and "sufficient evidence to act."

For example, if the threshold is set to 0.75 and the calculated posterior probability is 0.70, the evidence suggests the hypothesis is plausible, but not strong enough to justify action yet.

The appropriate threshold depends on the context and the cost of being wrong.

• Exploratory analysis: 0.60 – 0.70
• Policy decisions: 0.70 – 0.80
• High-risk decisions: 0.80 – 0.95

Higher thresholds require stronger evidence but reduce the risk of acting on incorrect conclusions.

Not all evidence has the same reliability. The Confidence Matrix helps users estimate the overall quality of an evidence source by scoring three dimensions.

The system averages these scores to generate the final confidence value used in the Bayesian update.

Three Evidence Quality Dimensions

• Source Reliability
  Measures how trustworthy the source of the information is.
  • Rumor / informal claim → low reliability
  • Official statistics or verified data → high reliability
• Sample Size / Method
  Evaluates how the evidence was collected.
  • Anecdotal observation → weaker evidence
  • Structured surveys or peer-reviewed studies → stronger evidence
• Recency
  Considers how current the information is.
  • Outdated data may no longer reflect present conditions
  • Recent observations typically provide stronger signals

The average of these three dimensions becomes the final confidence score applied to the evidence.

Effective LR = LR ^ confidence

This means low-confidence evidence still contributes to the analysis, but its impact on the final probability is reduced.

Each evidence item displays its impact on the posterior probability.

This shows how much the hypothesis probability changed after the evidence was incorporated into the calculation.

• Positive impact → evidence supports the hypothesis
• Negative impact → evidence weakens the hypothesis

This feature helps analysts quickly identify which observations are driving the final result.

The Top Drivers section highlights the pieces of evidence that contributed the most to the final probability update.

This allows users to quickly identify the key factors influencing the analytical conclusion.

Large positive values indicate strong supporting evidence, while large negative values indicate strong counter-evidence.