P.S. Every year you work is a year you don't have to "cover" with your retirement. And you'll likely need some health insurance, I'm lucky to have retiree insurance and now Medicare.
I do generally think that "betting on you" as others have put it is probably a good idea and worth at least seriously considering but, being the resident accountant/pessimist I'll give some devil's advocate thoughts. I'm doing this largely because a lot of people have already said "go for it" so me being another one of those wouldn't really help you so I'll take the opposite side.
The item I quoted from
@Cincydawg is an important consideration. You are my age (late 40's) so you have ~30 years to the average life expectancy. IIRC you said your 401k had around $1/2 Million. If you can add $33k/yr and you stay three more years you get to $600k plus you have three less years to cover.
Math:
- $500k/30 = $16,667/yr
- $600k/27 = $22,222/yr
- $5,555/yr difference 33%
That is WAY oversimplified as it doesn't account for investment returns either while you are working or after you are retired but what I am illustrating here is that three extra years can make a VERY substantial difference.
A second consideration:
How cyclical is your business? Badge mentioned being heavily involved in housing and having to trim his company from 28 employees to 6. Housing is hyper-cyclical because it can more-or-less always be deferred so in a recession it tends to plummet. OTOH, if you are selling low-cost food that tends not to be cyclical at all because everybody has to eat. Some businesses are even counter-cyclical as demand grows for lower-cost items when recession worries peak.
Personally, I think we are headed for at least a slow-down. That said, there is an old joke that most economists have predicted 5 out of the last 3 recessions because they always tend to predict recessions, LoL.
I didn't predict the 2020 Pandemic. I didn't predict the 2008 credit crunch/recession. I didn't predict the early-2000's dot-com bubble burst. Thus, I'm not going to pretend that I can predict the next calamity. That said, I can predict with absolute certainty that there WILL BE a calamity. It might be just a garden-variety recession or it might be some weird credit-thing or it might be some weird pandemic but whatever it is, there WILL BE a calamity.
If your business is hyper-cyclical, I wouldn't want to depend on it until I got to the point where I could comfortably live without it.
A third consideration:
How hard would it be to get back into employment if your side business went sideways? You mentioned making low $100's and considering that to be bad. You also said you don't have a degree. That is fine if you are a certified underwater basketweaver or whatever and your profession is in demand and you could either stay certified or easily get recertified and return to that line of work if you had to. If not, the salary for non-degreed labor around here is nowhere close to $100k. If you had to jump back in, where would you realistically be jumping in?
I don't have a lot of faith in SS and frankly, at our age I don't consider $500k to be anywhere close to enough to live on for the rest of our lives. Assuming a 10% return and 30 years of remaining lifespan that $500k is enough to get you around $53k/yr. Is that enough for you to live on? Furthermore, that rate may be optimistic and even if you hit it, you'd be flat broke at 78. If you stretch that out to 40 years the annual drops to $51k, is that enough? If you stretch it out to 40 years and think you need $100k, you need almost $1M.
At this point in your life and using my 10% assumed return, you need to add $2,763,26 to add one more year to your retirement "coverage". So if you work one more year and put $2,763.26 in your account, you are effectively adding two years. (That relationship isn't linear just FYI).
Just food for thought and I'm (mostly) just playing devil's advocate.