There's nothing preventing you from building a system with less than a kilobyte of on-chip memory in L1 and L2.
Whether such a system is suitable for your use case, depends on your use case.
If a monitor is throwing that assertion, it means some device is trying to access invalid physical memory. You should trace down exactly what hardware/software is causing this. Perhaps there is a simple software fix.
If you are using L1 DCache as scratchpad, and that is the bulk of the memory in your system, you should make sure the DCache is configured as direct-mapped. IIRC, only a single way of the cache can be used as scratchpad.
-Jerry