Why is it so hard? Because it needs to be relevant to the storyline, not offensive, and sound natural to the person speaking it, taking their age, time, where they live, and other demographics in mind. You need to listen to how people speak.
Yet, many people, especially those the ages of middle grade characters, have said little to nothing in my presence. Yup—people watching is tougher than you think, excluding the risk of those folks thinking that you’re stalking them. You could watch movies too, but that doesn’t really help, either. Another option is to read books and see how other authors write their characters’ dialogue.
But the hardest challenge with dialogue, overall, is having characters react believably to extreme situations, especially in fantasy. I write fantasy and I cannot stress enough how difficult it is to make characters react naturally to high levels of danger. No matter how hard I try, readers have said that the characters’ reactions were muted, unnatural, and too accepting. It’s so frustrating!
However, I found a solution, besides receiving help from editors. I print out the story and read the dialogue out loud. I was surprised to discover how unnatural some lines were—just by reading them out loud. So I changed the words.
Observing others is fine up to a certain extent. Also, a lot of people are quiet in public. Many even put on faces in public and might behave differently in their homes. Reading other books could work, as well. But I find reading the dialogue out loud helps the most.