[Python] Removing frame/outside of the text from red LED dotted display

I am building a MAUI app and need to preprocess photos of a red dotted LED display so OCR can read the digits. The photos often include a bezel/frame, screw holes around the LEDs. I must remove that noise (top/right/left/bottom depending on the photo) while preserving every LED dot. Many preprocessing attempts either leave frame/screws or destroy the dot grid.

I can only use ImageSharp / .NET features (EmguCV / OpenCvSharp are not an option for iOS in my project). Below is the current ImageSharp class I’m using, it roughly follows: grayscale → blur → edge detection → large dilation → threshold → invert. That pipeline could not remove the noise properly.

public class ImageSharpOcrProcessor
{
public static byte[] ProcessImageForOcr( string inputImagePath, byte binaryThreshold = 64 )
{

// Load image

using var image = Image.Load<Rgba32>( inputImagePath );

return ProcessImageInternal( image, binaryThreshold );
}

public static async Task<byte[]> ProcessImageForOcrAsync( string inputImagePath, byte binaryThreshold = 64 )
{
return await Task.Run( () => ProcessImageForOcr( inputImagePath, binaryThreshold ) );
}

private static byte[] ProcessImageInternal( Image<Rgba32> image, byte binaryThreshold )
{
// Convert to grayscale
image.Mutate( x => x.Grayscale() );

// Gaussian blur (5x5 kernel, sigma = 0 means auto-calculate)
image.Mutate( x => x.GaussianBlur( 2.0f ) ); // radius ~2 approximates 5x5 kernel


// Canny edge detection (ImageSharp doesn't have built-in Canny, so we'll use edge detection)
//image.Mutate( x => x.DetectEdges( KnownEdgeDetectorKernels.Sobel ) );
//alternative
ApplyCannyLikeEdgeDetection( image, 20f, 80f );

// Dilate (ImageSharp doesn't have built-in morphological operations, so we'll implement)
for ( int i = 0; i < 4; i++ ) // Apply dilation twice for stronger effect
ApplyDilation( image, 4 );

// Apply binary threshold before inversion (best practice for OCR)
ApplyBinaryThreshold( image, binaryThreshold ); // Convert gray pixels to pure black/white

// Bitwise NOT (invert)
image.Mutate( x => x.Invert() );

// Encode to PNG bytes
using var memoryStream = new MemoryStream();
image.Save( memoryStream, new PngEncoder() );

return memoryStream.ToArray();
}

// Alternative enhanced edge detection method (closer to Canny) - OPTIMIZED
private static void ApplyCannyLikeEdgeDetection( Image<Rgba32> image, float lowThreshold = 50f, float highThreshold = 150f )
{
// Apply Sobel edge detection directly (skip extra Gaussian blur since we already did it)
image.Mutate( x => x.DetectEdges( KnownEdgeDetectorKernels.Sobel ) );

// Apply simplified threshold for better OCR results - OPTIMIZED
image.ProcessPixelRows( accessor =>
{
for( int y = 0 ; y < accessor.Height ; y++ )
{
var pixelRow = accessor.GetRowSpan( y );
for( int x = 0 ; x < pixelRow.Length ; x++ )
{
var pixel = pixelRow[ x ];
// Use only R channel since image is already grayscale (faster than averaging RGB)
float intensity = pixel.R;

// Simplified threshold: strong edges become white, everything else black
// We'll let the binary threshold step handle final cleanup
byte newValue = intensity > lowThreshold ? (byte)255 : (byte)0;

pixelRow[ x ] = new Rgba32( newValue, newValue, newValue, 255 );
}
}
} );
}

// Custom dilation implementation for SixLabors.ImageSharp - OPTIMIZED
private static void ApplyDilation( Image<Rgba32> image, int kernelSize )
{
int width = image.Width;
int height = image.Height;
var originalPixels = new byte[ width * height ]; // Use 1D array for better performance

// First pass: copy original pixels (only R channel since grayscale)
image.ProcessPixelRows( accessor =>
{
for( int y = 0 ; y < height ; y++ )
{
var rowSpan = accessor.GetRowSpan( y );
for( int x = 0 ; x < width ; x++ )
{
originalPixels[ y * width + x ] = rowSpan[ x ].R;
}
}
} );

int halfKernel = kernelSize / 2;

// Apply dilation with optimized kernel
image.ProcessPixelRows( accessor =>
{
for( int y = 0 ; y < height ; y++ )
{
var pixelRow = accessor.GetRowSpan( y );
for( int x = 0 ; x < width ; x++ )
{
byte maxValue = 0;

// Optimized kernel bounds
int startY = Math.Max( 0, y - halfKernel );
int endY = Math.Min( height - 1, y + halfKernel );
int startX = Math.Max( 0, x - halfKernel );
int endX = Math.Min( width - 1, x + halfKernel );

// Check kernel area with optimized indexing
for( int ny = startY ; ny <= endY ; ny++ )
{
for( int nx = startX ; nx <= endX ; nx++ )
{
byte pixelValue = originalPixels[ ny * width + nx ];
if( pixelValue > maxValue )
maxValue = pixelValue;
}
}

pixelRow[ x ] = new Rgba32( maxValue, maxValue, maxValue, 255 );
}
}
} );
}

// Binary threshold implementation - converts grayscale to pure black/white for better OCR
private static void ApplyBinaryThreshold( Image<Rgba32> image, byte threshold = 128 )
{
image.ProcessPixelRows( accessor =>
{
for( int y = 0 ; y < accessor.Height ; y++ )
{
var pixelRow = accessor.GetRowSpan( y );
for( int x = 0 ; x < pixelRow.Length ; x++ )
{
var pixel = pixelRow[ x ];
// Use R channel since image is grayscale
byte intensity = pixel.R;

// Apply binary threshold: anything above threshold becomes white (255), below becomes black (0)
byte newValue = intensity > threshold ? (byte)255 : (byte)0;

pixelRow[ x ] = new Rgba32( newValue, newValue, newValue, 255 );
}
}
} );
}
}


Sample input / output Input image (raw):



Current output (what my pipeline produces):



I can't rely on a fixed crop (I previously cropped 25% from the top) because images vary in distance/angle/lighting.

I have tried several ImageSharp-based approaches (blur → Sobel/edge → dilation → threshold, etc.) but they either leave frame or destroy the LED dot grid.

I want a robust method (or a short Python/OpenCV proof-of-concept I can port) that automatically removes bezel/frame/screws from any side (top/right/left/bottom) while preserving all LED dots.

And I also try to deskew to straighten the text. But no success as well

Continue reading...