Maybe change your hand position just a bit so you don't dig in so deep? Are your fingerpicks straight or curved? If straight, try bending them up toward the fleshy part of your finger and see if that helps. Move your banjo neck up and down and see if that helps.
In other words, experiment with different ideas and see if one does the trick.
I notice you are the originator of the electric banjo thread down the page--I almost brought it up down there--that is another of my abstract realizations--it may or may not --fit your question=
anything hooked up to pick up the bridge or the head under the bridge--picks up string noises really bad--that type of sound wave though much weaker gets there first-- It is heard first and that location is too "sensitive"I use pick=ups of both types but I custom install to avoid that and to get them buffered physically
my original thinking -I ask no one to adopt or agree
Try to strike the 4th string flat on as much as possible. The 4th string always has a slightly different acoustic sound; get used to it. You may have to change the angle of your right hand for a second or everytime that 4th string has to be played...Jack p.s. I personally wouldn't rotate my picks too much as it can cause tendon damage over the years. I know that some Pros do it but they started at a much younger age and learned how to do it without hurting their hand by twisting their wrist a little....Jack
When moving to the 4th string the index pick can favor the 'thumb edge" giving that "get out of jail" filing sound. If index on the 4th isn't too often a little change in hand position might suffice.If this is a common practice a slight rotation of the pick might work better.