I say this because in any work of fiction, the world shouldn't just be a background setting, like some painted backdrop your characters walk around in front of. The world shapes your characters, and your characters shape their world. So even in a highly character-driven story, the world will be an important part of the story, even if your readers don't notice it. Nail down those details, work out how the world, your plot, and your characters interact, and your story will be much richer and more believable.
For the rest of this post, I'm mostly going to be discussing world-building in fantasy (and to a lesser degree, SF. Not because SF doesn't need bucket-loads of world-building thrown in, it usually does, but just because of my personal bent towards fantasy.) For myself, a fictional world usually starts with a single idea or spark. It might be something I see in a movie, or read on a blog, or hear in a lecture theatre. You can get ideas from damn near anywhere, as long as you have the idea receiver in your brain tuned in. You might get a story idea first, but for me I usually begin to develop a world first, then work out what stories could be told in it.
For example, the world for my next WIP came from a mash-up of ideas from:
1) a dream I had about a leather-coated train travelling through snowy mountain passes and
2) another idea in my notebook surrounding a natural environment that had been damaged by a technologically advanced colonising force trying to pass themselves off to the natives as gods or magicians.
The world began to grow because I kept asking questions. Why is the train leather-coated? Maybe when the "magicians" tried to demonstrate their powers, they accidentally released a toxin into the environment that rapidly corrodes metal. Machines and metal weapons would be useless, unless adequately protected.
What is the nature of this toxin? Is it chemical? Biological? Airborne? Waterborne? Ah ha, perhaps it is an airborne biological agent, which is why the train can only operate high in the mountain passes. It's too cold and too high for the spores or whatever to significantly affect the train.
As you continue to ask yourself these questions, more details come to light. Then you can ask yourself about the implications of these new developments on the world, and so on. As you come up with new ideas, feel free to chuck them into the mix and see if they mesh well. If so, great! If not, write it down in your ideas notebook for another story. This process is fluid; things can and should be shifted around, questioned, added to or eliminated. You may even change the entire plotline of the novel. That's fine. Go with it. It's still early stages.
All right, so you've got yourself a nice little world developing. While I'm working on this, I like to start thinking about how the people or other creatures of this world fit in. The world I'm using in my example is fairly forbidding, so humans would be confined to settlements in places where the effects of the biological toxin are less marked. How are these settlements formed? Are they independent city-states, or part of an empire? Ruled by committee or by a ruthless overlord, crushing all beneath his leather-covered boot? How do the common people view their lot in life? Do they resent their leaders, hate their environment? Are they oppressed, or free to do as they choose? What does a normal person do for a job?
When you start getting into the practical side of your civilisations, the most important things to think about are often the essentials of life. Food, water, shelter. Who controls the food supply? He who controls the food, controls the people. Is water easy to come by, or is it a precious commodity? Do people gather together in skyscrapers for shelter, or are they nomads, sleeping under the stars?
One of my favourite podcasts for thoughts on this is the Writing Excuses podcast: Writing Practical Fantasy. Check it out.
So now you've got the practical side of your society sorted out, it's time to consider other aspects. Culture is essential in any society. Culture is not something foreign, it is something that every single one of us has, whether we realise it or not. A white, middle-class person has a culture just as a tribesman in an undiscovered tribe in South America has culture. Understanding how your society's culture operates and how it affects your characters is essential. This can be as wide-ranging as the ways in which a family is formed (nuclear family? polyamory? matriarchal?), to the ways religions interacts with the population and with each other. Was your culture formed after generations of isolation, or were dozens of different cultures thrust together, trading and assimilating each others' cultural quirks?
These are just a few questions I ask myself early in the world-building process. If you want a great list of things to consider in your world-building, I recommend the SFWA World-building resources. They ask you to think about things we've talked about here, along with many others.
About now, many writers (me included) fall prey to world-builders' disease. Somewhere around the time you're working out the evolutionary history of the algae in the pond outside the Chief Mage's tower and the meaning of the name of the King's auntie's third cousin's pet rock, you realise you should probably stop all this world building and actually WRITE THE DAMN BOOK. So put all that world-building into a handy folder and get started. It's ok if you don't have every aspect of the world worked out. Really.
Just fake it.
You heard me. Details are the key. Your readers don't want to read everything you created in your world. They want to read a story about characters. If you provide enough detail to make that world seem real and interesting, they will be happy.
So get your butt in that chair and build that world. Then stop building that world, and get stuck into the real work.