Modern calculus (analysis) was invented because people shot themselves in the foot working with topology and wondering exactly what is a "curve" ? I am a big fan of this approach to learning mathematics, just forge ahead and when (if) things fall apart then go back and fix up the foundations. To this end I recommend a couple of books. "The Knot Book" by Adams is a very interesting exploration in topology (without requiring all the years of study at university before you are allowed to learn exactly what a topology is). And in another direction, group theory was invented because the study of symmetry gets very tricky! But if you want to dive in anyway then have a look at Conway's "The symmetries of things". It is a lot of fun. Most modern group theory (or algebra) books don't actually have any pictures of symmetric things, just endless formulas and lemmas. If you want to be a pro, then you gotta learn that stuff, but there's definitely pathways into higher mathematics that don't require you to learn that.