The idea has been forming in my mind for a while now and I’m unsure if a project exists yet or at least one that is free. The idea is that there are laser usability testing and plain video testing. What about web cam usability testing? I know the precision will be less than laser, but if you want the El Cheapo brand usability testing, then it is better than nothing right?
All you would need are:
- Buying a Web Cam or two. Kind of a requirement if I’m going to be doing testing and development. I can’t blindly code this.
- Facial Recognition
You do need to know the placement of the eyes. Also if the head moves as it is prone to do, then you need to be able to tell.
- Calibration
I’m guessing is a lot less tricky when dealing with math and precision to simply tell the viewer to stare at a location on the screen for a moment and get the five or more points on the screen. This will give you a sense of where the edges are and the center given the position of the head. May have to “Lock” the head in place. The only problem is that it is easier to tell if the head moved directional but harder (I think) when dealing with depth. I’m thinking that depth could be calculated based on the width of the face, the other problem is telling if the head is turned.
- Trigonometry
The key is transforming the 2D space into a 3D space and back to a 2D space. The camera to the eyes and back to the screen. This creates a triangle which should correlate to a position on the screen. It should be possible to calculate the position by having the person look at the screen. If you know where the person is looking at, it should be possible to calculate the distance of the camera to the screen. Even if it isn’t possible, then it doesn’t really matter. By getting the points of the screen, you can get the position of the screen based on where the eye might be looking. By sampling the head, the ears (if visible), the nose, the chin, mouth and eyes to get the position of the head given the screen.
Sure, it is probably cheating, but if it is accurate enough and fast enough, then I don’t think any one is going to complain. I’m definitely not going to pester the person to get a ruler or tape measure to measure the web cam to the center point. However, it might be an option if precision is a factor.
- Accessing the Web Cam
I’ve learned that you do this by DirectShow. This might limit the amount of programming languages and Operating Systems. I’m definitely not going to support Linux, but I may look into Mac OS X support. It will be interesting to find out how to program for that.
- Accessing the Screen Content
The key is to let the person running the usability testing to know where the person was looking and running the test back with what the person looked at and when. My goal also is to let the person know where the mouse went as well.
- Play Back
Well, it isn’t useful if it isn’t recorded. What I might attempt to do is record the screen along with the user and do post-processing with what the user might have been looking at. This would allow for improving the algorithms for facial recognition and precision of where the user might be looking at.
- Research and Development
It would be fun I think to work on the AI algorithms and math. I’ll be brushing up on my trig skills and hope to hone my math ability during this project. I’m hoping the precision isn’t terrible and that this can actually be used for what I’m wanting. If not, then I might fall back to taking pictures on an interval and screenshot too. Not going to be as precise as a video running 30 frames per second, but I think it might be much easier to track real time if it was only a single frame a second. Also, the resolution of most web cameras are higher than the video resolution which should hopefully allow for even higher resolution.
Why I Probably Won’t Do This
The damn complexity of it all.
The facial recognition will be hard enough and I will work on that first and work on the math for the depth and what not. I’m hoping that it won’t be as difficult as I’m fearing. If I can work out the head position when it moves in relation to the camera and, or screen then I’ll be at least part way there. It’ll be fun I think depending on how long it takes to implement the facial recognition.
I’m also hoping to remove a lot of the tricky Trig and simply have guess the position based on the calibration I’ll have the user do beforehand. I’ll probably develop a overlay for testing the software to display where I think the user is looking at. This will also help with testing the precision. If the precision is enough that I can tell apart the word, ‘it’ from the ‘i’ and the ‘t’ then I think it will be good enough for production. If I can half ass the Trig and still get the precision within the requirements then it will be a win-win. I will still have to spend some time learning Trig and other math subjects to prepare.
Patents. I know they must exist, but I’m not sure what they are. I’m merely guessing that since Photoshop and every camera maker out there can intelligently locate the face that at least that part is patented. I’m also guessing that this idea isn’t new and companies have developed technology implementing this idea. Trending on that money maker probably is going to get me sued. Regardless, I’ll still attempt to the best of my ability to create this and research existing patents as I develop. Perhaps if I’m super nice, I’ll be able to use the patents without incurring a bankrupting huge cost.
I’m super lazy and I have another 3 or 4 projects I want to implement. Depending on my motivation, I’ll probably work on them first.
Why I Want To Do This
Well, Selenium is great for browser testing. However, while that is great. I think it will be equally great if someone would create an easy to use User Usability Testing software like they use in labs, but far cheaper. If the goal is to do usability testing often, then what better way than using cheap equipment and a whole lot of people. If you could use anyone with a web cam, then you could even do so remotely.
I did have plans to somehow have the developer see the user as the person is doing the usability testing. I think that going through Skype or uploading the finished product should be sufficient for the amount of effort and time I’m planning on putting into this project.
Of course it will be open source and of course I’ll probably be violating a crazy amount of patents to implement this. The later part does kind of suck, since I will lack the funds to license any patents I’ll be infringing on and also that there are damn patents in the first place! Ah well, I’m hoping that the former will at least allow a better programmer to come along and use a method that isn’t patented.
Possibly Related Posts:
- Game Engine Development and Open Source
- Plans for Base CMS
- Project Plans
- Calibre Improvements Part 2
- Motivations for Programming
Comments are closed.