Both are correct, but have slightly different meanings.

It's a hot, sunny day (ha ha, schön wär's) and you get up thinking "I'd rather like..." Heute würde ich gerne..(in etwa).

Still a hot, sunny day - and someone suggest going out to see a movie: I'd rather go...Ich würde lieber...

They don't mean the same thing.

In the first sentence, 'rather' means 'quite', as in 'It's rather cold today'. So it means: 'I'd quite like to go swimmming.'

In the second, it means 'rather than something else', i.e. 'I'd prefer to go swimming.'

I'd rather like to go swimming sounds BE to me.

Sounds like perfectly normal AE to me.

I'd rather like to go swimming sounds BE to me.

sounds weird to me... rather than dislike?

In my experience, the phrase "would rather like to" in the sense of quite (see #2) is uttered in the US only by those strongly influenced by BE. It makes me think of BBC costume dramas.

I would quite like to go swimming also sounds BE to me. Both would be perfectly understood, of course.