In a context like gardening or the back yard, I would say 'dirt' is the normal, most common word -- not especially colloquial, just everyday. (*f5* In AE, at least.) If your grass doesn't grow evenly, you might have patches of bare dirt where the dog runs along the fence, or under the trees where there's not enough sun. If it's rained recently, the ground might be a little muddy and people and pets might track dirt inside if they don't wipe their feet. Children often get dirt on their clothes when they play outside; parents might tell them not to play in the dirt. If it's very muddy, you might see the tracks of wild animals in the dirt. An unpaved road in the country is called a dirt road; an off-road bicycle or motorcycle can be called a dirt bike.
'Ground' can be the flat dirt, but it's also the flat surface, no matter what material forms it. People with common sense have their feet on the ground. Electrical wires have one wire called the ground that goes to the ground (AE; but BE 'earth,' I believe). Pilots who can't fly are grounded; so are kids who have to stay at home as a punishment. Tree roots grow below ground, and caves are below ground. You can dig a hole in the ground, for a fencepost or a tree or a grave. If you're outside and you need to put something down that you're holding, you can set it on the ground.
'Soil' describes the type or quality of dirt, often in a scientific sense. You might do a Ph test to see if the soil in your yard is acidic or alkaline. Poor soil might need additives like compost or minerals. Soil scientists do research that helps farmers and conservationists.
When you repot plants, you might use potting soil that you buy in a bag at the garden store. In a flat region like a prairie, strong winds can blow away the topsoil, as in Oklahoma in the 1930s, which made it into the Dust Bowl.
'To soil' as a verb can also be a fancy word for 'to (make) dirty.' Upper-class people might not want to soil their hands with hard work.
Both 'soil' and 'earth' are also used in more formal or literary contexts, like poems and hymns. We speak of heaven and earth, or earth and sky.
Abel became a herder of sheep while Cain was a tiller of the soil. And it happened in the course of time that Cain brought from the fruit of the soil an offering to the Lord.
'Earthen' is an adjective meaning made out of dirt or earth. Earthenware is a kind of pottery. A dirt floor could also be called an earthen floor; you could have an earthen dam, or earthen dwellings, built out of mud or dirt, as in Africa.
Of course 'earth' also means our planet, this world. There are around 7.8 billion people on earth. We celebrate Earth Day in April. In the words of an oft-quoted poem, pilots and astronauts have 'slipped the surly bonds of earth ... and touched the face of God.'
Hope that helps. (-: