Archaic or not, forbye
is still used now and then outside the Scottish dialect. Some fairly recent instances, prepositional and adverbial (in addition to the Kenneth White quote in the original post):
Forbye imagined allusions to saltire, thistle, and clootie dumpling, there was perhaps something irresistible about virile angularity supported on swelling curvature, implying among other things that in this outfit, the mechanistic depended on the organic.
Herald Scotland, 18 Sep 1996
“Why could you not have started earlier, instead of wasting half the morning like this? Do you think we can afford to pay good wages for men to stand around doing nothing? And, forbye, why do they have to walk? Why not take the wagons right up to the pithead?”
extract in the Scotsman, 13 June 2016
The Mudhook Yacht Club, by virtue of the fact it was set up to have ''40 members and one forbye'' may be small and exclusive, but it has gained a cherished reputation for promoting and aiding excellence in regatta and rules management.
Herald Scotland, 8 Aug 1998