If the downward motion of lifting a weight transfers energy to you, why does your arm get tired?
Your body is unable to store working that’s done on it and also wastes energy even when it is not doing any work. When you lower a weight, the weight does transfer energy to you, but your body turns that energy into thermal energy. You get a little bit hotter. If you were made out of rubber, you might store it as elastic potential energy (like a stretched rubber band). Instead, your muscles don’t save the energy in a useful form. As for getting tired, your muscles turn food energy into thermal energy even when you aren’t doing work. That’s what happens during isometric exercises. There’s nothing you can do about it. It’s like a car, which wastes energy when it’s stopped at a light.