The BBC website had an explanation about how they do this:-
The scammer says he's ringing from, say, your bank, and asks you to ring your bank and ask for him by name to check him out.
So, you put the phone down, and call your bank.
What happens is - surprise - you get through straight away, so...