Like Google Translate, the program is designed to learn over time. As long as its continually fed more and more translations, Conversation Mode will presumably get better and better at its job. This might mean that more obscure languages and those less often translated by Google will take a little longer to fare as well as other languages in Conversation Mode. However, this is an application designed to improve over time, and it's not unreasonable to conjecture that in the future Conversation Mode will be able to handle nearly any speech it encounters. Another beautiful example of cloud-based computing at work.
This may not be as easy as putting a Babel Fish in your ear, but it's far easier than scrambling through a pocket dictionary and taking minutes to ask where is the restroom? And we can probably expect Google to keep Conversation Mode free like it's done with its other applications. No date is announced, but watch for this to become publicly available soon.
Watch this video of the app in action from consumer electronics trade show Internationalen Funkausstellung Berlin, known as the IFA, this past September: