How to Use BERT for High-Accuracy Semantic Search in Python

Bytewebster
Date : April 23, 2025
Copy Link

How to Use BERT for High-Accuracy Semantic Search in Python

In the world of web development and data science, the importance of semantic search has grown significantly. Unlike traditional keyword-based search, semantic search aims to understand the meaning behind the query, delivering more accurate and contextually relevant results. One of the most powerful tools to achieve this is BERT (Bidirectional Encoder Representations from Transformers), a state-of-the-art model developed by Google.

In this blog, I’ll walk you through how to use BERT for high-accuracy semantic search in Python. The best part? You don't need to be an expert in NLP (Natural Language Processing) to follow along. I'll make it simple and clear for you.

What is BERT and Why Should You Care?

Before jumping into the technical side, let’s understand what BERT is and why it's so useful for semantic search.

BERT is a transformer-based model that has revolutionized NLP tasks. Unlike older models, BERT reads text in both directions (left-to-right and right-to-left), allowing it to capture more context and nuances in language. This is what makes it so powerful for tasks like semantic search, where understanding the meaning behind the words is crucial.

When we apply BERT for semantic search, we’re essentially using its ability to convert text into meaningful embeddings (vector representations). These embeddings can then be compared to find the most relevant search results based on their semantic similarity.

What You’ll Need:

To get started, you’ll need a few things:

Python: Make sure you have Python installed. If not, download it from the official site.
Transformers Library: Hugging Face’s Transformers library makes using BERT easy.
Sentence Transformers: This is an additional library that helps you create sentence embeddings.
Pandas: For data manipulation (optional, but recommended for managing your dataset).

You can install the required libraries using pip:

pip install transformers sentence-transformers pandas

Step 1: Load the BERT Model

First, we’ll load a pre-trained BERT model using Hugging Face’s Transformers library. This is super easy because Hugging Face has made it simple to work with BERT and other NLP models.

from transformers import BertTokenizer, BertModel

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')

Here, bert-base-uncased is the version of BERT that doesn’t differentiate between upper and lower case letters, which is often more than enough for most tasks.

Step 2: Convert Text into Embeddings

Now that we have the BERT model, we need to convert our search queries and documents into embeddings that BERT can understand. Let’s start by encoding the text using the tokenizer and passing it through the model to get the embeddings.

import torch
def get_bert_embeddings(text):
    inputs = tokenizer(text, return_tensors='pt', padding=True, truncation=True)
    with torch.no_grad():
        outputs = model(**inputs)
    embeddings = outputs.last_hidden_state.mean(dim=1)
    return embeddings
text = "How to use BERT for semantic search"
embedding = get_bert_embeddings(text)
print(embedding)

In the code above, last_hidden_state.mean(dim=1) averages the token embeddings across the sentence to get a single vector representation for the entire text.

Step 3: Comparing Text Embeddings

To perform semantic search, we need to compare the embeddings of the search query with the embeddings of the documents in our database.

We’ll use cosine similarity to measure how similar two embeddings are. The closer the cosine similarity is to 1, the more similar the two texts are in meaning.

Here’s how you can compare two text embeddings:

from sklearn.metrics.pairwise import cosine_similarity
def calculate_similarity(embedding1, embedding2):
    return cosine_similarity(embedding1, embedding2)

query_embedding = get_bert_embeddings("How to use BERT for semantic search")
doc_embedding = get_bert_embeddings("Semantic search is a powerful tool for understanding meaning behind text.")

similarity = calculate_similarity(query_embedding, doc_embedding)

print(f"Cosine Similarity: {similarity[0][0]}")

Step 4: Putting It All Together – Building a Simple Semantic Search System

Now that we know how to get embeddings and calculate their similarity, let’s put everything together to create a simple semantic search system.

Imagine you have a list of documents and you want to find the most relevant one for a given query. You can follow this process:

Encode all the documents into embeddings.
Encode the search query into an embedding.
Compare the query embedding to all document embeddings using cosine similarity.
Rank the documents by their similarity score and return the most relevant ones.

documents = [
    "Semantic search uses AI to understand the meaning behind the text.",
    "BERT is a powerful model for NLP tasks.",
    "In this tutorial, we will learn how to use BERT for semantic search.",
]

doc_embeddings = [get_bert_embeddings(doc) for doc in documents]

query = "How does semantic search work?"

query_embedding = get_bert_embeddings(query)
similarities = [calculate_similarity(query_embedding, doc_embedding) for doc_embedding in doc_embeddings]

ranked_docs = sorted(zip(similarities, documents), reverse=True)
print("Most relevant document:", ranked_docs[0][1])

Step 5: Enhancing the Search with Sentence Transformers (Optional)

To make your semantic search even more powerful, you can use Sentence Transformers. This library is built on top of BERT and fine-tuned for generating high-quality sentence embeddings that work even better for tasks like semantic search.

Here’s how to use it:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')

sentence = "How to use BERT for semantic search"
embedding = model.encode(sentence)
print(embedding)

Sentence Transformers are optimized for tasks like semantic search and will give you even better results than the basic BERT model.

And there you have it! You’ve learned how to use BERT for high-accuracy semantic search in Python. By converting text to embeddings and comparing them using cosine similarity, you can create a powerful search system that understands the meaning behind the words rather than relying on simple keyword matching.

BERT might seem intimidating at first, but once you break it down, it’s actually quite simple to implement. Whether you’re building a search engine, a recommendation system, or just playing around with NLP, BERT is a tool that can take your projects to the next level.

I hope this tutorial helps you let me know if you run into any issues, or if you have any questions, I’m always happy to help!

How to Use BERT for High-Accuracy Semantic Search in Python