- Published
- Updated
Boolean Retrieval: AND, OR, and NOT in Search
Learn how Boolean search uses AND, OR, and NOT to combine result sets, where exact control helps, and why website search still needs ranking.
Search does not always need to infer a user’s meaning. Sometimes the user knows which words are necessary or not permitted.
For example:
api AND authenticationinvoice OR receiptwebhook AND NOT legacyThese are Boolean queries. Boolean retrieval uses a match-or-no-match model.
We will use a small documentation site as an example. First, Boolean operators select the matching pages. Then, the ranking system orders those pages.
What is Boolean retrieval?
Boolean retrieval finds documents that satisfy logical conditions.
Each query term has a set of matching documents. Boolean operators combine those sets:
ANDkeeps documents that occur in both sets.ORkeeps documents that occur in either set or both sets.NOT Bkeeps documents outside set B in a defined search universe.A AND NOT Bremoves set B from set A.
Boolean operators give users direct control when they know which documents are acceptable.
Boolean retrieval defines the matching logic. The search engine defines the query syntax, supported operators, and scoring behavior.
A five-page documentation site
All examples use the following five pages.
A full-text search system usually analyzes page text before it searches. It converts selected text into indexed terms.
The system also converts the query into compatible terms. It then gets the document set for each query term.
For example:
Raw page text: "Create API keys"Indexed terms: api, keyMatch set for key: {D1, D2, D5}The Complete indexed terms column contains all indexed terms for each page.
| ID | Page | Representative text | Complete indexed terms |
|---|---|---|---|
| D1 | Create API keys | Create API keys; authenticated request. | create, api, key, authenticated, request |
| D2 | API authentication | API authentication; bearer token; key. | api, authentication, bearer, token, key |
| D3 | Verify webhook signatures | Webhook authentication; signing secret. | webhook, authentication, signing, secret |
| D4 | Retry failed webhooks | Retry failed webhook delivery. | retry, failed, webhook, delivery |
| D5 | Legacy API key migration | Legacy API key migration; deprecated. | legacy, api, key, migration, deprecated |
The example analyzer does these operations:
- It converts uppercase letters to lowercase.
- It removes punctuation.
- It reduces simple plurals such as
keystokey. - It does not stem other words or add synonyms.
These are the match sets:
api -> {D1, D2, D5}key -> {D1, D2, D5}authentication -> {D2, D3}webhook -> {D3, D4}retry -> {D4}legacy -> {D5}The search engine can get these sets from an inverted index . The index records the documents that contain each term.
AND means intersection
AND requires both conditions. It does not require adjacent words. Exact phrase matching is a separate feature.
Use this query:
api AND authenticationThe api set is {D1, D2, D5}. The authentication set is {D2, D3}.
AND keeps their intersection:
{D1, D2, D5} ∩ {D2, D3} = {D2}Only D2 matches. D1 does not contain authentication, and D3 does not contain api.
Now use this query:
webhook AND authenticationThe two sets are:
webhook -> {D3, D4}authentication -> {D2, D3}Their intersection is {D3}.
AND usually makes the result set smaller. Each added condition removes documents that do not satisfy it.
An AND condition can make a broad query more precise.
OR means union
Boolean OR is usually inclusive. A document can satisfy the left clause, the right clause, or both clauses.
Use this query:
api OR authenticationThe sets overlap at D2:
api -> {D1, D2, D5}authentication -> {D2, D3}OR combines the sets:
{D1, D2, D5} ∪ {D2, D3} = {D1, D2, D3, D5}D2 occurs once in the result set. D1 and D5 match api, and D3 matches authentication.
OR usually makes the set larger. It is useful when two terms are acceptable alternatives:
invoice OR receipterror OR failureremove OR deleteThe query lists the acceptable alternatives. Do not assume that the engine knows they are related.
NOT means exclusion
NOT starts with a defined search universe. It keeps the documents outside the excluded set.
Our complete universe is:
U = {D1, D2, D3, D4, D5}This query removes the legacy set:
NOT legacyThe result is:
U - {D5} = {D1, D2, D3, D4}Now use:
api AND NOT legacyThe api set is {D1, D2, D5}. Remove D5 because it contains legacy:
{D1, D2, D5} - {D5} = {D1, D2}Some query languages permit api NOT legacy. Others require api AND NOT legacy.
Some languages do not permit a standalone NOT query. The parser grammar controls the valid forms.
The universe can be one site, project, language, or collection. Without this boundary, NOT legacy has no complete set to search.
Parentheses show the intended order
A query can contain several operators:
api OR webhook AND authenticationThe query has two possible meanings.
The first meaning is:
(api OR webhook) AND authenticationapi OR webhook matches all five pages. The authentication condition reduces this set to {D2, D3}.
The second meaning is:
api OR (webhook AND authentication)webhook AND authentication matches D3. The final union is {D1, D2, D3, D5}.
The two groupings return different document sets.
Many Boolean systems apply NOT before AND and AND before OR. Do not assume that every product uses this order.
Parsers can use different precedence rules, case rules, or symbols. Parentheses show the intended grouping.
Use explicit parentheses for mixed operators. Also, use the syntax in the product documentation.
Results from the example corpus
| Query | Matching pages |
|---|---|
api AND key | D1, D2, D5 |
webhook AND authentication | D3 |
api OR authentication | D1, D2, D3, D5 |
api AND NOT legacy | D1, D2 |
NOT legacy | D1, D2, D3, D4 |
(api OR webhook) AND authentication | D2, D3 |
api OR (webhook AND authentication) | D1, D2, D3, D5 |
Text analysis changes the input sets
Boolean matching usually operates on analyzed terms, not direct character strings.
An analyzer can do these operations:
- Convert text to lowercase
- Split text into tokens
- Remove punctuation
- Reduce words to a common root
- Remove common words
- Expand synonyms
- Apply language rules
Our analyzer changes API keys to api and key. Thus, key can match a page that contains keys.
The analyzer does not connect authenticated to authentication. Therefore, D1 is absent from the authentication set.
A different stemmer can produce this larger set:
authentication -> {D1, D2, D3}The Boolean logic is unchanged. AND still calculates an intersection. The analyzer changed the input sets.
Analyzer: Which indexed terms match this query clause?Boolean logic: How should the match sets combine?You must know both parts to explain a Boolean result.
Text clauses and structured filters are different
A Boolean text clause searches analyzed content:
api AND keyA structured filter checks exact field values or metadata. This example is pseudocode, not portable syntax:
section = "guides"version = "v2"language = "python"Text clauses can search titles, headings, and body content. Filters can check product, version, language, category, date, or workspace.
A request can use both:
Text query:(api AND key) AND NOT legacyFilters:version = "v2"section = "guides"The text query checks analyzed terms. The filters restrict the candidates with structured data.
Authorization is a mandatory system constraint. User query syntax must not weaken it.
For one user, the search universe must contain only permitted documents. A user-controlled NOT private clause is not access control.
Useful cases for Boolean retrieval
Boolean retrieval works well when the user knows precise terms.
Require an important term
webhook AND authenticationThis query removes general webhook pages that do not discuss authentication.
Accept two names
member AND (remove OR delete)The alternatives can find pages that use either verb.
Exclude marked old content
api AND key AND NOT legacyThe query works only when legacy is a reliable marker. Structured version data is safer when it is available.
Search exact technical terms
Boolean clauses can combine error codes, function names, product names, fields, and protocols.
ERR_AUTH_42 AND webhookThe analyzer controls whether ERR_AUTH_42 stays one term. Use the engine’s exact field and syntax when the complete identifier is necessary.
Build advanced search controls
An interface can provide these controls:
Must containMay containMust not containThe interface can convert them to Boolean clauses. Users do not need to learn the raw syntax.
Candidate selection is not ranking
Boolean retrieval answers one question: Which documents can match?
Boolean retrieval does not always answer a second question: Which matching document must appear first?
The query api AND key matches D1, D2, and D5. Basic Boolean logic treats all three as matches.
A ranker can then use term frequency, title matches, or other evidence to set the order.
BM25 is one method for lexical ranking. A system can enforce Boolean conditions and then rank the permitted documents.
Some engines also let scored Boolean clauses affect the order. Check the documented behavior of your engine.
Limits of Boolean retrieval
Boolean retrieval gives control, but the user must select useful terms.
This query can be too strict:
api AND authentication AND bearer AND rotationThe result is empty if no page contains every analyzed term.
This query can be too broad:
api OR authentication OR request OR tokenCommon terms can admit many weak candidates.
NOT also needs care. python NOT legacy removes a current page that only mentions a legacy comparison.
Boolean retrieval does not solve all vocabulary differences. remove teammate can miss a page named Delete workspace member.
An analyzer, synonym rule, or explicit alternative can connect those terms.
Search engines can also support phrases, proximity, field rules, wildcards, and fuzzy matching. Boolean operators do not define those features.
Summary
Boolean retrieval combines document sets:
ANDcalculates an intersection.ORcalculates an inclusive union.NOTexcludes a set from a defined universe.- Parentheses show the intended order for mixed operators.
- Text analysis determines which documents enter each term set.
- Structured filters operate on fields and metadata.
- Authorization defines a mandatory permitted universe.
- Boolean logic selects candidates, and ranking puts them in order.
In our five-page corpus, api AND NOT legacy returns D1 and D2. D5 contains api, but the legacy condition removes it.
Boolean retrieval has two steps. First, build the match sets. Then, combine them with explicit logical rules.
Next reads
What Is an Inverted Index?
An inverted index maps terms to the documents that contain them, helping search engines find text matches without scanning every page.
What is BM25? A Plain English Guide
BM25 is a keyword search ranking function that scores documents using term frequency, term rarity, and document length. Learn how it works, where it helps, and how it differs from TF-IDF.