Here’s a post I’ve been meaning to write for a while, but just never got around to it.
As a Waterloo Engineering student, we have to complete a final year design project under the guidance of a supervisor in groups of three.
What have I been making?
My project is called Peekaboo. It uses facial detection and recognition to allow a user to find out more about people around them.
I’ve been working on this for about 5 months now with my group members. We’ve made sufficient progress and are now at a stage where I can talk about our systems architecture and some of the design decisions that we made.
System Overview
Peekaboo is comprised of 3 components, shown in the diagram below.
System Overview
iOS App + Face Detection + Parse
We have a native iOS app that uses Parse for it’s mobile backend. We’ve been very happy with Parse, and it was painless to set up. Face detection is performed on the device itself through Face.com’s API. We went with Face.com over OpenCV’s Face detection due to higher accuracy and the availability of more data. Face.com is able to give us JSON back with eye positions, mouth posiitions, head tilt and yaw, gender, and more.
I strongly believe user experience should be important to any project. We use a lot of great open source iOS UI components available at Cocoa Controls.
Web Server on Heroku
We wrote a web server in Node.js deployed over Heroku to act as an API. Once again, Heroku makes this painless to set up and deploy. The web server is the mediator used by the iOS app and the desktop C++ code. The web server talks to Parse to get information and feeds it to the C++ code (and vice versa). It uses standard authentication.
The web server has a MySQL database which we usually query instead of directly querying Parse. This way, we stay below the rate limits. Parse is only queried when there is a face that needs recognition.
- We found Sequelize to be a good ORM for MySQL (available as an npm module)
- We found Request to be a useful module for simplifying HTTP Requests.
Facial Recognition Algorithms
I won’t lie, I’m not the best at C++. I find web languages and Objective-C a lot easier to code in. Nonetheless, when doing Face Recognition, we had to use C++. It’s mainstream and fast. Of course, we went with OpenCV, one of the most popular computer vision libraries out there (See my post on how to set up OpenCV on Mac).
Like I said, we don’t use OpenCV for Face detection. We use it to work with images, do some basic image processing (histogram equalization) and use OpenCV’s Eigenfaces implementation. We have also implemented another Facial Recognition algorithm known as LDA (Linear Discriminant Analysis). This was done mostly through reading research papers and contacting people smarter than us. Google Scholar is a useful resource. LDA gives us a 10-15% improved recognition rate over EigenFaces.
Communication Layer
Since I’m not a big C++ guy, I wrote a pretty thin Objective-C layer on top of the C++ code that communicates with the web server and figures out when to execute the facial recognition code. I was able to leverage a lot of my iOS knowledge for this, using standard libraries such as ASIHTTPRequest and SBJSON (ASIHTTPRequest has recently been stopped being maintained, but for our project, it does the job).System diagram of how facial recognition works
Final Thoughts
Some parts of our code are a little hacked together. This is partly due to the time crunch we are under and partly due to the fact that we are still learning. However, I am pretty proud of what we have achieved so far. We can accurately (over 70%) recognize a person who has been trained by our system (database size is 70 people currently). We’ve used $0 on this project, which is a testament to how open and amazing our software engineering industry is. We have learned a lot through out this process. I’ll put another post up with a live demo sometime soon.
If you have questions, ask them below!
