Some people just have a sluggish metabolism. This means that even when you eat less, you struggle to lose weight. Age can also work against you.