It may be brake dust material that you are smelling. You only smell it when you apply the brakes right. How many miles on your brakes? I would suggest removing the drums to inspect the amount of remaining brake shoe material and to check the wear pattern. If they are disc brakes its a bit easy to inspect. Also make sure your rear e-brake is not hanging up. The only rubber components on the brake system are seals ie wheel cylinder, axle seals etc. Another thing ..... drum brake tend to carry a fair amount of brake dust in them. This will also be a possiblity . Don't blow the dust out with an air hose. The dust is hazardous. Use a water hose to rise out the dust. Let us know what the out comme is.
Joe