Ok I'll bite, you do present an interesting case! If the DZ has some good rental gear for downsizing, maybe you can at least convince them to allow a rapid downsizing. Back in the old days, early in the zero-p canopy era, if you were good you might do a couple jumps on a particular size canopy, then downsize a size, and repeat. If going to a sportier canopy, have a few jumps on a more docile canopy of the same size. Anyway, that was my experience in the early 1990s as someone who also thought they were a special case -- a pilot in my case.
Try to borrow rigs as well, if needed to help with the downsizing. At least then you might have an idea what works for you before you spend money on gear yourself.
If you're already on a 180 at 2 jumps that's just one 1 to 2 more downsizes to a 150 anyway.
I'm not a great source of advice as I don't speedfly (only paraglide). I'm not sure of the glide ratios current speedflying or riding canopies are built for, but I'd warn you to be careful of the more ground hungry skydiving canopies until you worked your way into them. At the same wing loading, there can be canopies of widely varying flight characteristics. I guess they aren't really common at say 150 size, but at 135 and under they start to be more common. They have a steep descent (say, glide ratio of under 2.5) and dive sharply in turns. So be careful of the model of canopy and not just size.
Whether you end up buying a 150 or whatever, it is just very difficult to know what to look for (in a rig, reserve, and main) when you've only barely started skydiving. You could probably fly and land smaller than a 150 easily enough, but you would also be learning to deal with packing, body position on opening, and dealing with other skydivers on opening and in the landing pattern. So who knows, maybe a 150 at 1.25 loading or thereabouts might be reasonable to stay at for a little while.