A few options exist.
Possibility 1: You've got a companion with a terrible sneak skill. This is the least likely.
Possibility 2: Your sneak skill is low. This is highly likely.
Possibility 3: You're wearin lots of heavy armor. Armor, especially heavy armor, makes you easier to detect. For the best results wear special stealthy armor, robes, or just underwear. This will definitely help, but in the end you need to upgrade your sneak skill.