De-cluttering your screenshot images in the gallery with AI
Ever taken a screenshot of an important transaction to share with your friend? Or maybe to capture an important map, articles, invoice, medical report, funny meme, or a conversation. Be it anything, eventually, after a while, we tend to forget to remove these images from our phones and as they gather dust and clutter our gallery, they also become dangerous weapons when in the wrong hands.
At times, these can contain personal information which when in the wrong hands can cause us inconvenience or big problems- Think virtual credit card information
May we should spend a day decluttering and removing PII data from my phone. But over time, it would gather again. Being an engineer, I thought to myself, only if I could categorize my screenshots in the gallery in an organized way, keeping things until necessary and then deleting them when no longer required. So here’s what I did —
Custom Vision in Azure
This Azure service allows users to push bunch of images as sample data for image classification or object detection. Here’s how you can get started.
DIY Guidelines -
Create your own project on the custom vision portal and upload sample data that summarizes your gallery. (choose project type as classification)
Label/tag the images into major categories that you observe consuming most space in your screenshots album/gallery folder.
Add negatives label to handle images that don’t categorize into any of the major categories
Train the sample data with classification model as a single tag per sample and publish that iteration to generate a prediction endpoint
Example Iteration result-
I have classified the images into Personally identifiable information(PII), Maps, and stock market. The sample data is small for representation in this example. However, with time and each prediction, we expect the data to grow and the AI predictor to be more and more precise.
Now, we have an AI model, and all we need is this service to be exposed to be used in Android/iOS apps or simple console applications. Also, the custom vision SDK makes it easy to create a quick test application. I have uploaded a python test script here.
Quick Test -
New image prediction output -
As shown, the image is identified based on sample data which includes invoices, payment transactions, contact details etc.
The sample application can be improvised to run on entire directory of images to categorise and tag the images at one go.
