Should I break up with that really shy guy and move on?

Now me and my bf (who is really shy) were dating for about a month now. I want to break up with him but I'm afraid that Ill regret it. The reason I want to is because he never answers his phone, and when he does he just says hi and then says he will call me later witch he never does. He doesn't make me feel special in anyway. Whenever we r going on a double date with other people, he never shows up so I'm always a 3rd wheeler. Four of my friends are telling me it would be better if i broke up but one is telling me not to. I'm also afraid that we wouldn't be friends after the break up. What should I do??