and yes, jump's fine!

Well you could "jump in the pool" but you would have to be in the water to do so. Imagine a pool that isn't too deep, you're in this pool, standing on the ground and jump - that would be a jump in the pool. (Wo springe ich? Im Pool)
If you're not yet in the water but want to be, you jump into the pool. (Wohin springe ich? In den Pool)

"I'm gonna go jump in the pool and cool off."

"I am about to jump into the pool."

Both work. Saying "jump in" may not technically be correct, but most people at the pool do say it, at least in my experience.

"Jump in" is perfectly OK, but "jump into" may be appropriate in some contexts.

It depends on how important the process is, compared with the result (or something like that).

"Go jump in the lake" (piss off), but

"He jumped into the pool from the high diving board.".