(in case anyone feels like they came in on the middle of the conversation, rockbassist (aka Kevin) asked a question over
here.)
Kevin, how much of your interest is in moving more into jazz, versus (for example) doing less structured extended jams in a rock setting?
One of the things I find curious is the whole concept of studying, or teaching someone, how to be creative. At first blush, that just seems wrong - yet there are approaches that seem to work, to varying degrees, for some people.
I like what 811952 (John) said above. Most instructional materials on improvising start you off with lots of review of scales, theory, getting to know your instrument thoroughly, and so forth. I have a couple of old books, The Improviser's Bass Method by Chuck Sher, and Concepts for Bass Soloing by Marc Johnson, and that's exactly how they start out (and go on at great length, more than I had the patience for...).
But the real key seems to be, as John said, to listen and play the intervals and phrases you hear in your head, rather than think about a half-diminished run beginning on the and of one.
Yes, you have to know your instrument very well. Whether or not that requires exhaustive understanding of theory, being able to instantly play any named arpeggio in all possible positions, etc. is questionable in my mind. Certainly this stuff is important, and the more you know the better - but it can also be sort of a trap, because the whole art of improvisation is being able to get beyond this stuff, to the point where you can just play what you hear in your head, without thinking.
I think of it a little like developing muscle memory, where your fingers learn how to play some particular scale without thinking about which frets to aim for, except that you need to take that a couple of levels higher, so that you don't have to even think about the progression (and maybe not even know what it is). More like a muscle memory that extends directly from what you hear or imagine in your head, directly into the strings (in our case) without any conscious translations.
I think that's why a lot of the suggestions here are along the lines of playing along with music you don't know. It frees you up from trying to follow a chord chart, dredging up all those rules you learned about how to transition from one to the next, which particular scales to play over each... you just listen, and do it. If you already have a decent familiarity with your instrument and want to work on improvising, then the last thing you should do is go back to studying scales. Maybe pick one and jam on it for a while, but try not to think about it.
I recently watched a DVD that was largely an interview with Keith Jarrett, who may well have spent as much time as anyone on the planet thinking about improvisation. In the last few years, he has shifted his approach, for both practicing and performance. He used to improvise lengthy pieces (30-40 minutes or so), and now favors shorter things. When practicing, he says he now stops himself as soon as he finds he is playing an old familiar pattern, to force himself to break out of habits, like stringing together the runs his fingers know so well (how he can still find anything new after all this time is a bit of a mystery to me...).
There's also a great comment on this stuff by Bill Evans in a short interview on one of his recordings, but it's a little too long for me to transcribe right now. The gist of it is that it does involve discipline and practicing, but the key is to get to the point where you can just throw the switch and it happens.
Enough for now, I'm supposed to be working...
-Bob