It sounds like your puppy wasn't very well socialized in his first few weeks of life. It will just take more patience on you and your family to make this little guy feel comfortable. You must give him a few days to come around, doodles adjust very quickly and I'm sure he will come around with a lot of love and patience. Don't rush things along, let him explore and get use to all his new surroundings. He has a lot to take in. Enjoy this time with your new puppy and he will soon be rewarding you with lots of kisses and a lifetime of devotion 
