Graph Neural Networks for Image Similarity: An Alternative to Hashing?

Traditional hashing techniques like perceptual hashing (pHash) and locality-sensitive hashing (LSH) are widely used for image similarity detection. However, they often fail in real-world scenarios where images undergo transformations such as cropping, noise addition or color changes.

Graph Neural Networks: A Smarter Approach to Image Similarity

Graph Neural Networks (GNNs) provide a more robust alternative by modeling image relationships as a graph and propagating similarity information through message passing. Instead of treating images as isolated entities, Graph Neural Networks (GNNs) represent images as nodes in a graph where edges define their relationships. This allows context aware learning making similarity detection far more robust.

Let’s talk about a Real world Needs

Consider an AI driven content moderation system that should detect inappropriate images across billions of uploads daily. If a flagged image is slightly altered/cropped, filtered or resized traditional hashing fails allowing harmful content to evade detection.

\ Now imagine an e-commerce search engine that helps customers find similar products. Two shirts with different backgrounds but identical designs might have very different pixel values but should still be recognized as the same product.

\ In both cases a GNN-based approach would outperform traditional hashing by understanding relationships beyond pixels using graph structures to connect similar items dynamically.

Building an Image Similarity System with Graph Neural Networks

Let’s walk through a tutorial to build GNNs on any image set.

\

1. Extract Image Features Using CLIP or ResNet:

   Before constructing a graph, we extract feature embeddings from images using a pre-trained model

import torch import torchvision.transforms as transforms from PIL import Image from torchvision import models #Load a pre-trained model model = models.resnet50(pretrained=True) model.fc = torch.nn.Identity() model.eval() #Image transformation transform = transforms.Compose([ transforms.Resize((224, 224)), transforms.ToTensor() ]) #Load and preprocess an image def extract_features(image_path): image = Image.open(image_path).convert('RGB') image = transform(image).unsqueeze(0) #Add batch dimension with torch.no_grad(): features = model(image) return features.numpy().flatten() feature_vector = extract_features("my_image.jpg")

\

2. Construct a Graph from Image Embeddings

   We can now construct a KNN (K Nearest Neighbor) graph where each node represents an image and edges indicate similarity.

import networkx as nx import numpy as np from sklearn.neighbors import NearestNeighbors #Simulated feature vectors for 100 images (assuming 512D embeddings) image_embeddings = np.random.rand(100, 512) #Build a KNN graph k = 5 nbrs = NearestNeighbors(n_neighbors=k, metric="cosine").fit(image_embeddings) distances, indices = nbrs.kneighbors(image_embeddings) #Construct the graph G = nx.Graph() for i in range(len(image_embeddings)): for j in range(1, k): # Skip self-loop G.add_edge(i, indices[i, j], weight=1 - distances[i, j]) #Visualize graph structure print("Graph nodes:", G.number_of_nodes()) print("Graph edges:", G.number_of_edges())

\

3. Train a Graph Neural Network (GNN) for Similarity Learning

We now use PyTorch Geometric to train a GraphSAGE model on this graph.

import torch import torch.nn.functional as F from torch_geometric.nn import SAGEConv from torch_geometric.data import Data #Convert the NetworkX graph to PyTorch Geometric format edge_index = torch.tensor(list(G.edges)).t().contiguous() x = torch.tensor(image_embeddings, dtype=torch.float) #Define a GraphSAGE model class GraphSAGE(torch.nn.Module): def __init__(self, in_channels, hidden_channels, out_channels): super().__init__() self.conv1 = SAGEConv(in_channels, hidden_channels) self.conv2 = SAGEConv(hidden_channels, out_channels) def forward(self, x, edge_index): x = self.conv1(x, edge_index) x = F.relu(x) x = self.conv2(x, edge_index) return x #Initialize and train model model = GraphSAGE(512, 256, 128) #Reduce embedding size to 128D optimizer = torch.optim.Adam(model.parameters(), lr=0.01) for epoch in range(50): optimizer.zero_grad() out = model(x, edge_index) loss = F.mse_loss(out, x) #Self-supervised learning loss.backward() optimizer.step() print(f"Epoch {epoch+1}, Loss: {loss.item()}")

\

4. Querying Similar Images Using the GNN Embeddings

Now that training is complete, we can retrieve the top k most similar images for a given query.

from sklearn.metrics.pairwise import cosine_similarity #Extract GNN embeddings with torch.no_grad(): learned_embeddings = model(x, edge_index).numpy() #Compute cosine similarity query_idx = 0 # Example query image similarities = cosine_similarity([learned_embeddings[query_idx]], learned_embeddings) top_k = np.argsort(-similarities[0])[:5] print("Most similar images:", top_k) Final Thoughts

GNNs create a smarter and more adaptive approach to image similarity. Unlike hashing, which relies on rigid pixel comparisons, GNNs learn meaningful relationships between images, making them far more robust to transformations, occlusions, and noise.

\ In today’s world where image datasets are exploding in size from social media platforms to medical imaging archives and the rise of synthetic data, scalability and accuracy are critical. GNNs provide an efficient way to structure and retrieve similar images improving applications like visual search, content moderation, recommendation engines and AI driven media management.

\ The future of image search isn’t just about finding duplicates, it’s about understanding the deeper connections between images, and GNNs take us one step forward to that a reality.