I've installed numerous hardwood floors both in my home and in homes of family. The few floors I've done I've been lucking enough that they fell within the manufacturers tolerance for being flat. I'm now installing hardwood in a walk-in linen closet I built and the floor has a deviation of 1/2 inch from the high spot to low spots.

I'm putting down 3/4 hardwood that will be nailed down so I can't use the self-leveling mixes. I'm not thrilled with using roofing shingles inside as my wife has concerns with outgassing. Question is: can I use cedar shims to fill in the gap under the wood and level the floor and would the shims go under or over the vapor barrier?

 

1/2" is a lot to try and shim each and every board and certainly more than what is normally fixed by felt/shingles.

If it were me, I'd be working on the floor with additional layers of plywood to get the flooring closer to level then using conventional leveling with felt/shingles.

If out gassing is a concern they now make silicon paper that is used as underlayment in place of felt, a few shingles is not going to ever be noticeable!

 

I used level compound under nail down floor five years ago and had no problems with it. I got some from flooring dept. But have been using some concrete finisher on my very pitted driveway that is a little cheaper but says for flooring. Nails had no problem with going through leveler.

 

Somethings very wrong under that floor to be that far out, I'd be checking under that area to see what's causing it.
Just adding more plywood over it is not going to fix anything when it's that bad.

 

You could use cardboard (Non corrugated type)

All depends on how large of an area need shimming.