Face recognition and jpg Metadata Tagging – using Facebox and ExifTool
I have been working on a fun little project this week. Like most people, I have thousands of unorganized photos on my home network, they are in random folders all over the place. I know I have photos of weddings, birthdays, hikes, vacations, etc.. but they are buried in random places all over my network. Sometimes they are in a folder with a descriptive name like ‘Moab 2008’ but sometimes they were dumped in a hurry and ended up in a folder like ‘crap from phone.’ Last week I was in charge of making a wedding video for my sister in law, I know have lots of pics of her, but I was searching through folders for at least an hour and only came up with a few. The face grouping on Google Photos helped a little, but some of these photos have been around a lot longer than Google Photos. I need some way to organize my photos locally. That is where Facebox comes in…
Facebox is a piece of software made by MachineBox (https://machinebox.io/). They claim to be super easy to use and be able to start recognizing photos in just a few minutes. I have actually wanted to test this out for a couple months, but it seemed to be an intimidating project. I thought I would have to learn a bunch of complicated stuff before I could actually use it. Turns out it actually is pretty easy to use. All you have to do is send two POST commands to a webservice end point.
The first command is to teach a face:
curl -X POST -F ‘file=@/home/jaron/jaronteach.jpg’ ‘http://localhost:8080/facebox/teach?name=jaron&id=jaronteach.jpg’
The second command is check a photo for faces:
curl -X POST -F ‘file=/home/jaron/test.jpg’ ‘http://localhost:8080/facebox/check’
Turns out I could have been playing with this quite a while ago. They said it was easy, and it is crazy easy… If you know how to send POST requests to an endpoint then you can get started with it.
The data it gives you back is just a block of JSON.
{ "success": true, "facesCount": 1, "faces": [ { "rect": { "top": 160, "left": 77, "width": 185, "height": 185 }, "id": "jaron1.jpg", "name": "jaron", "matched": true, "confidence": 1 } ] }
If you can filter through the JSON then you can start doing some pretty slick stuff. I used Python for this project and the JSON library for Python made this crazy easy:
if facecount>0: for face in photodata['faces']: if face['matched'] ==1: person=face['name'] print("This photo has "+person+" in it! I'm gonna do stuff with it!") if face['matched'] ==0: print("This photo has an unknown face in it! I need to befriend this person!") else: print("This photo does not have any faces in it, time to check out tagbox..")
I had some basic logic and a way to identify faces in photos quickly. Now I just needed a method to organize them. I started out moving pics of my daughter to her own folder, and my son to his own folder, and my wife to her own folder, etc.. I still like this idea, but I am not totally sold on it. I had accounted for pics with more than 1 person in them by just copying the image to both of their folders; intentionally creating a duplicate. I still don’t hate this idea, but I have a big family, I am going to be creating duplicates everywhere. Time to look at metadata tagging… That is where EfixTool comes into play.
EfixTool is made by a dude named Phil Harvey (https://www.sno.phy.queensu.ca/~phil/exiftool/). This tool allows you to add metadata to an Image file…. that’s right, it adds it to the file. You can’t add custom fields, but you can add things to the windows ‘Tags’ field if you know which field to update with EfixTool 🙂
The beauty of this is that the metadata stays with the image, so if you get a new computer, or have multiple computers on your network, you don’t need a connection to some central database that has all your metadata tagging. The other benefit of it is that you can search these tags in Windows!
You can also sort your images in Windows by the tag:
So.. how do you update the windows tag field in command line? This was probably the thing I spent the most time trying to find.. and I did not actually find the answer in a forum post or a question someone answered, I found it on another efix site that listed every efix field available to update (huge thanks to this list right here – http://www.exiv2.org/tags.html). The one I was looking for is called XPKeywords
after downloading efixtool from (https://www.sno.phy.queensu.ca/~phil/exiftool/) you can start using it to update the metadata, this is actually pretty easy once you know what you are looking for.
This is how you add or replace metadata to that field via command line:
exiftool ‘/home/jaron/jaron.jpg’ -XPKeywords=zo
This is how you append more, after the initial data has been added (thanks to this post – http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=8120.0):
exiftool ‘/home/jaron/jaron.jpg’ ‘-XPKeywords<${XPKeywords};jaron’
The above syntax is for linux, i think windows may be slightly different, like double quotes instead of single quotes or something. If you are unsure what your current file has for metadata you can query a specific tag like this:
exiftool ‘/home/jaron/jaron.jpg’ -s -s -s -XPKEYWORDS
One other thing you might want to add to the exiftool commands is -overwrite_original. By default it will make a copy of your original files and name it something like image.jpg_original. By adding the metadata you are duplicating every file. This is easily avoided by adding -overwrite_original like this:
exiftool ‘/home/jaron/jaron.jpg’ ‘-XPKeywords<${XPKeywords};jaron’ -overwrite_original.
Does all this sound too easy? Yes, I probably did play it down a bit.. but not really.. If you can get docker installed, then you can install Facebox in a matter of minutes. You don’t even have to sign up for anything to use it! You just plug in your email and they send you a link to download it. I did have to spin up a Linux vm to install Docker and Facebox on, but that is only because I do not have the required Windows version for Docker. if you are running an enterprise version of Windows you should be good to go.
https://www.docker.com/
https://machinebox.io/
https://www.sno.phy.queensu.ca/~phil/exiftool/
About the author
2 comments
Leave a Reply Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Very cool! Make it into an app and you’d have gold.
Thanks Andy! it has been a fun little project.