Did you replace the internal wires when you installed the new connector? If not it is a quick inexpensive thing to try.
Otherwise since it is the cabinet it sounds like the voice coil might have a bad connection or a crack in it. As the coil moves it will warm up which would make the metal expand closing the problem area and making a connection. As it cools and the metal shrinks it opens up the bad area. An ohm meter put on the speaker might or might not prove this depending upon the type of connection/coil problem. If you try to measure be sure the speaker is not hooked up to the amp.
