I'm sorry for your loss.
My daughter was 2 weeks shy of 3 when my father died. I consulted with a children's grief counselor and part of her advice was to avoid euphemisms. They may not fully grasp the idea of "death," but it's better than a version of "gone away." It's extremely hard for a young child to comprehend that there's somewhere a loved one can go where the child can't follow and the loved one can never come back. It's already so far outside of their understanding, that you just shouldn't mince words at that point. Papa died. He didn't leave, he died. Death means that the angels came to show his spirit how to get to heaven and leave his body behind. His body is empty now, because his spirit is gone and can't come back. You can talk to his spirit, but he can't answer you back.
My daughter had the advantage of watching my father die, so while the concept of death in the abstract is still out of reach a year later, she had a concrete experience to draw on, but even then she'd ask me questions like "when is Papa going to come back so we can stop missing him?"
All you can do is give him straight answers and let him ask whatever he needs to ask. Allow him to participate in the mourning process as much as he's able, I think it really helped. I don't agree at all with shielding kids from mourning. Our children learn how to handle their emotions from watching us. They learn how to deal with anger constructively, how to cope with frustration, how to be a gracious winner and loser, grief is no different. Prohibiting them from taking part in the mourning process isn't going to make them not feel the loss, it's just going to deprive them of the option of learning how to handle it.
There is no way that a 3-year-old can understand death, or the concept of Heaven. We dealt with it when a family member died by telling our 3-year-old grandchild that her other gran had gone to be a star in the sky, and that's why she couldn't come back again. She accepted this totally, especially since we went out with her and she picked the star that must be granny....
Of course, this has been forgotten, but it meant a lot to her at the time.
She is six now, and has recently lost her dog, heard that we've been to a friend's funeral, and knows that our own dog is dying. She accepts the idea of death as something that happens when animals or people get old and sick, but she likes the idea of Heaven as somewhere where her dog has lots of fun. What worries me is that she says she can't wait to die, because she wants to go to Heaven, because it sounds so beautiful.
when a family member passed away from cancer after being sick for some time, i explained it to my son (who was almost 2 at the time) that his body got really very tired and he went to live with the angels for now.
obviously, your situation is a little bit different, given that your father in law's death was sudden. i would probably tell my son that his body got very hurt in an accident, so much so that he went to live with the angels so it wouldn't hurt him.
im a believer in being upfront and honest to my kids. they're kids, but they're not stupid! talk to him in private, but you may need to remind him when he asks because they need reassurance that the idea they have in their head is in fact, the correct one. be open when he wants to talk about it, regardless of who is around. his feelings on this matter are very important.
remember: he may not remember this event as an adult, but there is a chance he could. it could have long lasting effects on how he views death in the future, so be sure to be open and support any questions he may have, for as long as he may have them. its a confusing concept.
At three, he understands being hurt and being sick, so tell him an amended version of events. His grandpa had an accident, and his body was too hurt, and he could not get better. His spirit is in heaven, but he will always be with you in your heart.
i'm sure you know but don't say papa went to sleep for a long along time, etc. When my dad passed away, I just told my toddler that papa went to heaven with Jesus. Every time we go see mama, our son will say papa in heaven? And I'll say yes. We'll look at pictures and he'll give the pictures a kiss and I'll say that papa loves him. Ugh! Too sad.
try to explain in a way that the little guy will understand.Tell him God wanted papa in heaven with him.You won't see him but he will be able to look down from heaven and see you and guide you.He will miss you and grandma. Tell him papa is very happy being in heaven with God. Papa doesn't want anyone to be unhappy and cry because he is happy and doesn't want to become sad.
I don't at all envy your task of doing this...
Perhaps explaining what heaven is, or else explaining in another way saying he is in a happy place, and no longer in our world. Also you could tell him that although he will not see him, he is watching over him all of his life. The difficult question you have to answer is 'why can't I see him'.
You might ultimately have to explain what death is in order for him to understand. It might be the responsible thing to do instead of him thinking he is still around.
He still might not understand that until he is older. People often fall back on, 'he has gone far away into the sky, and you will see him again, but a very long time from now.
Well, you could just say: "Papa won't be coming, sweetie, not for a long time. He's gone to some place far away; up in the sky. You'll see him eventually, just not now."
Or something like that.
Hope I helped a bit. =)