PhoneFactor, session connects even if you don't answer phone call

I have an RDP Gateway set up that works with PhoneFactor.

One issue I have and I am not sure how to correct is that if you don't answer the phone factor call the RDP session is still established.

If you answer the call but do not press # it fails.

I looked around but I do not see where the setting would be to force the phone call to be answered, no answer no session. Seems useless if not forced to answer phone call.

 Thanks.