Don't penalize DEF for a pick-six or a fumble returned for TD.
A DEF should not be penalized for "points allowed" if they weren't even on the field when it happened. If Jared Goff throws a pick-six, it's hard to blame the LAR DEF for that. If Tom Brady gets strip-sacked and it is run back for a TD, how is that NE DEF's fault?
Frankly, I don't think the DEF should be penalized with "points allowed" resulting from the offense turning the ball over inside their own 20 yard line, but at least there is an argument to be made that the DEF was on the field when the points were eventually scored.
But, at the very least, a DEF shouldn't be marked down for points scored while they were never even on the field.
In your example, the NE D would not be charged for the 6 points scored. This is clearly stated in the rules of the game.
Click on the first bullet, and you will see:
Items that count towards "points allowed" are:
All FGs scored by opponent
All PATs (extra points and 2-PT conversions)
All TDs scored against defense or special teams (including fake punts / field goals / returns from kickoff, punts, blocked punts)
Items that do not count towards "points allowed" are:
TD returns from Interceptions, blocked field goals, or fumbles
They would be on the hook for the PAT.