Of course, it is not limited to VoiceXML use cases only, although this is what enterprises use.
You can use the Dialogflow API to the full extent with FreeSWITCH. Moreover, the results may optionally be passed and processed in JSON format, which makes the processing even easier from an FS JavaScript app.
When it comes to Asterisk, then you can use Generic Speech Recognition API (with res_speech_unimrcp) for conversational interactions (multiple recognition requests in the scope of the same session) and further play synthesized prompt back using MRCPSynth() application. However, the applications from app_unimrcp suite, such as MRCPRecog() and SynthAndRecog(), are currently not too much of use for building conversational interactions, since both apps establish a new MRCP session, perform a recognition, and terminate the session afterwards.