I looked at it years ago from the irvine side and wondered similar.
Presumably there was a way through across the Ardeer peninsula when it was built
I used to run from Stevenston to the Irvine Harbour but that was along the water line.
I did similar on a motorbike years and years ago.( I wondered if you could go from Saltcoats to Troon on the sand, the answer to that is yes BUT?
Following that type of path, at some point you would need to take a left toward Irvine town up a bit and then a right , to approach it from the top.
Google Earth will show you , most others had ICI blanked