This matter seems fairly simple, every time you change string gauge or height, you have to fine adjust your touch on strings. If strings are lighter or action was lowered compared to previous setup, we have to soften our way to hit the strings to prevent fret buzz (since tension is smaller now).
Also be sure checking neck relief to set the truss rod, in case of back bow (or excessive bow), since string tension will change too.