You definitely can start this way. When I started, I got some help from a physical therapist--I had really messed up my back, and in addition to helping with the acute issue, they also selected a set of exercises and numbers of reps for me that I could do at home, and that was a great starting point for my exercise routine. It was pretty short and focused, so it was easy to find time to do it every day, and the practice of keeping at it was really helpful. My health insurance covered most of the cost of the physical therapist; I had to pay a copay, but even then it was just a couple times a week for maybe two months, so not exorbitant. Insurance is generally willing to help with this stuff for a little while because they know that if your health improves, it's likely to reduce their future costs. So it's worth looking in to whether yours would help with something like that just to get yourself going. I don't think you need to have an acute problem to take advantage of that; I think having a specific goal for improvement is adequate. (They want measurable goals, like "I'd like to be able to jog five minutes without getting winded," or that sort of thing. I believe mine was "I'd like to be able to spend a day out walking around a garden with my family without being laid up the next day by my back." Which reflects where I was at the time. But, y'know, anything that reflects where you currently are, and something that you might be able to achieve in a six-to-eight-week timeframe, is probably a good goal.)
Doing that regularly also got me listening to my body, and that got me to gradually expand my routine--I eventually understood that some of my back issues were propagating up from hip issues, so now I work on those, and some of those are coming from limited ankle mobility, so I'm also working on that, and working on that has got me doing "goblin squats" that has gotten me to stop thinking of dumbbells as something to avoid. I'm also getting closer to being able to do pull-ups; I got a pull-up bar because just hanging from a bar sometimes can really help with a bad back, but at some point I started thinking about how much more I enjoyed moving when I was a kid and took gymnastics classes, and back then I actually had the strength to do things like pull-ups. So now I can do some resistance-band assisted pull-ups, and hopefully in a year or two I'll be able to do the proper thing.
Picturing enjoying movement is something that really motivates me, actually. Like, I used to enjoy biking and ultimate frisbee. I don't, now, but I think I might enjoy them again at some point. I think I might also enjoy parkour, if I can get into that kind of shape, but I recognize that may not be an achievable goal at this point. I had a kind of enthusiasm for brief bursts of very intense movement, like sprinting up a flight of stairs two at a time, or climbing up onto a loading dock in a single giant step.
At this point I do a basic set of dumbbell weight exercises, squats and lunges, push ups, a back stretching and exercises routine, assisted pull-ups, and a walking/running aerobics routine. It's not a ton, but I'm really in vastly better shape than I was when I started a few years ago. I do have a handful of equipment--the dumbbells, a floor mat, a couple of foam rollers, an exercise ball (for trunk lifts, which are good for a weak lower back), a doorway pull-up bar, some resistance bands that I basically just use with the pull-up bar, and the biggest thing is an elliptical machine for when the weather is too bad to do the aerobics outside. There are ways to do it without a machine, like jogging in place or doing rapid shallow squats, but the machine is kind of nice--it's hard to explain, but it really helps to have the exercise take place in its own little isolated space, or even just in its own mental space. I actually also have a little lighted sign that I made (it's a recreation of the neon sign for an exterminator's in my home town that always tickled my fancy back then--it's got a giant neon rat in the middle) and I like to turn that on in my room specifically while I do my exercises there (everything other than the aerobics and pull ups), just because it kind of marks out the distinction of exercise time. It helps make it a ritual, and that helps make it a habit. As I say, hard to explain, but it feels like it matters.
I will say, this routine has also helped me lose some weight. I'm down about 45 lbs (~22kg) from this time last year. That's mainly down to diet changes, but I did ramp up my exercising while doing this to be sure that I was losing fat rather than just losing muscle. I'm still a lot heavier than I'd like, but I'm definitely proud of how far I've come. I'm improving in other measures, too, like my resting heart rate is down from around 100 to around 80, which, again, is not where I'd like to be, but represents movement in the right direction.
So, I do think the physical therapist helped a lot with getting me started, but most of my work I've done at home, and without too much in the way of equipment.
Would I have done better, faster by going to a gym? I dunno. I definitely know that friction is a big factor. If it's hard to actually go do the thing, then it's easy to make excuses not to go do the thing; needing to actually travel to a gym definitely counts for that. There's kind of a balancing act in making my routine easy enough and pleasant enough that I'll actually do it, but also challenging enough that I'm still gradually improving. Sometimes I need to let myself slack off at something a little as an incentive do just do the thing. And sometimes once I'm actually doing the thing I don't need the slack after all.
Bit of a rant, I guess. Sorry, it feels like so much of this stuff is, like, techniques for outwitting part of my own brain, and it feels like those are things other people might be able to use, but I'm not sure how transferable they really are. Hope it helps.
Good luck with your journey! I know I'll need luck on mine.