So I've actually figured out a few things myself just continuing to hack away at it. Posting in case anybody else runs into this problem.
First, you have to copy the files (as Administrator normally) to C:\Windows\System32\winevt\Logs\. They should automatically inherit the permissions for that dir, but worst case you may have to set them to Full Control by Local Service.
Second, using PowerShell as Administrator, create a new event log with the same name as that filename (minus the .evtx). If you review the file with Event Viewer first you can probably see what source to assign. Ex: New-EventLog -LogName CustomApp -Source "Enterprise Library Logging Service". You should now see the file show up as an Application channel in Event Viewer.
Third, you create a localfile setting for the OSSEC agent to read from that event channel. Restart the agent and watch the logs to be sure that it subscribes to that channel.
Finally, the agent may not read the existing logs in the file and may start at the end. If that happens, stop the agent, find the bookmark file for it (normally in C:\Program Files (x86)\ossec-agent\bookmarks), and edit it with something Unicode safe. Change the RecordId back to 0. Save and restart.
The problem I'm seeing now is that the events are coming through to my OSSEC server (in logall mode) with many of the fields blank or showing the literal "none". I'm not sure why, but I suspect that it has to do with Windows reading from events that were not generated locally on that system. I'm not a Windows Event Log expert, though, so if anybody has any information I'd appreciate it.