One thing that helped me was having gentler induction techniques. I find that when I focus hard and become lucid, I have a bit more clarity but wake up soon after (no matter what I do to stay in). If I use something a bit gentler (like SSILD) I may have less mental acuity but the dream can last longer as well.