My friend from church has switched in the last year from full time teacher to part time special needs assistant. She loves it because she now has time for her children and husband as well. I can see how much happier she is. It's absolutely visible. So I hope this might help you decide. Life is too short to go around feeling guilty. Personally I believe family come first, especially children. After all, once they've grown up you can't get the years back again. I know that first hand as mine are grown up now. Don't waste your life being unhappy when you can change it by choice. I hope this helps.