Before there were any conscious organisms, there was the electromagnetic spectrum but not yet any eyes to discern particular colors and there were vibrating air molecules but not yet any ears to discern particular sounds. The world was actual but the world as perceived by us with blue sky, white clouds, green grass, invisible air, felt winds and heard sounds was merely potential. One actual material world must contain many potential perceived worlds, each corresponding to a potential organism with eyes sensitive to different wavelengths or with different kinds of sense organs.
Before there could be any conscious organisms, there had to be unconscious organisms. Therefore, we do not perceive the world as it was before perception became possible. There were necessarily three stages:
(i) a material world, none of it organic;
(ii) a material world with organisms, none of them conscious;
(iii) a material world with organisms, some of them conscious.
We perceive (iii) and infer the earlier existence of (i) and (ii).