Flicksync: Knights Who Say Ni - My first published game

[Unlimited Magic]

I created Flicksync to learn app inventor. It was a hard lesson but taught me much. Here's the result, enjoy!

Flicksync comes from the reference in the novel Ready Player One. There a boy plays the game in VR where he has to recite the exact line of a scene from the movie Monty Python and the Holy Grail. I thought I could do this with speech recognition playing the movie clip and pausing it to let the player speak the next line. Parsing the line to compare spoken phrases was a challenge because speech recognition doesn't always get everything right. So I had to test the phrases when I spoke with perfect diction and then use those results as the test targets. Then I realized that if you didn't get all the words exactly right, the score would penalize you for missing the entire phrase. So I compared the result with each word in the target phrase and assign a percentage of accuracy. If you don't get the phrase right at all, you lose a life. Just like in the novel, if you get 4 phrases right, you generate another "cue card". It took a while to make the app as bulletproof as I could. 

I went through many techniques in how to play/pause video clips but none worked right. Eventually, I just cut the scene video into tiny clips and played each one. This way I was sure it wouldn't pause at the wrong frame and allowed the scene to wait until speech is recognized at the same point on all devices. The file size of the app was too big so I had to add the video clips after the fact. To test the game, without the clips - I used still frames as placeholders.

Flicksync on Google Play