We tried Harry Potter but the 4- year-old absorbed incredible detail about insignificant plot elements (!) and wasn't able to pick up the larger scene-setting and story arc. The 7-year-old just wasn't into the magic aspect of it all.
We read Dragon in the Sock Drawer, Dragon in the Driveway, The Little Prince, and the first two Harry Potter books before he turned 5. Although he bounces all over the place while I read, and often asks me to explain things (and clearly missed what's going on at times he understands much more than I had expected, has been. Also, my son still likes to take breaks between reading chapter books. My tendency is to want to start a new one as soon as we're done, but he will often want to read some of his older picture books, and I try to follow his lead, while introducing the chapter books gradually. And the 5yr old loves the more involved stories, and is excited to be in charge of the bookmark. I haven't explored too many books yet, so I'm looking forward to other responses you get. As for falling asleep while listening - ask them the next night 'what do you last remember? Where do you want me to pick up?' So, in other words, start now!