Having taken my 3rd gen off-road, it did have sufficient ground clearance for the road I took it on--it's only about an inch lower than a first-gen. And a lift kit would easily make up the difference if I were keeping it much longer.
But, the bigger issue with CR-Vs is their weak engines and weak suspension parts. They are fine for city driving and mild off-road use (down a dirt road, mild surface irregularities, etc.), but the CR-V is already underpowered, and asking it to climb even a mild incline from a near stop is asking too much of it. I got stuck on minor things that the CR-V almost couldn't power its way over. One of the most stress-inducing was a switchback coming out of Shaffer Canyon--I barely got up the incline, and it had to be taken slowly since getting a running start would have sent me sailing over the edge of the road into the canyon.
And actually, it's not so much the engine as it is the gearing. Anything that goes off-road really needs the low-range gearing you find in something like a 4Runner or Tacoma (2H/4H/4L gearing...like any real off-roader has). So it's not just about ground clearance--I had ground clearance. I did not have the gearing. It was still a great trip, but would have been a lot less stressful if the CR-V's weak engine would make it get out of its own way.