I do this kind of thing *every day* in CMUCL!! It's my primary
user-mode hardware debugging tool. Here's a typical utility function:

    ;;; Used for things like polling for a "done" flag.
    ;;; BUG: Assumes contents of ADDR will change within fixnum polls.
    (defun spin-until-change (addr)
      (declare (optimize (speed 3) (debug 0) (safety 0)))
      (loop with v0 of-type (unsigned-byte 32) = (r32 addr)
            with counts fixnum = 0
            while (= v0 (r32 addr))
         do (setf counts (the fixnum (1+ counts)))
         finally (return counts)))

+---------------
| SIOD actually has (%%% memref address) for peek.
+---------------

CMUCL has SYSTEM:SAP-REF-{8,16,32} and SETFs of same. The R32
above is just my convenience wrapper around SYSTEM:SAP-REF-32:

    (declaim (inline r32))

    (defun r32 (addr)
      (declare (optimize (speed 3) (debug 0) (safety 0)))
      (system:sap-ref-32 (system:int-sap addr) 0))

    (defun w32 (addr &rest values)
      (declare (optimize (speed 3) (debug 0) (safety 0)))
      (loop for i fixnum from 0 by 4
            and v of-type (unsigned-byte 32) in values
        do (setf (system:sap-ref-32 (system:int-sap addr) i) v))
      (values))

Most other Common Lisp implementations surely have something similar.

-Rob