The pictures you're blitting onto them - are they like tiles in a tile
map (
http://en.wikipedia.org/wiki/Tile_engine)? If so then yes, you're
going about it the hard way :-) The wikipedia page may have some
insights. Otherwise you now have a useful google search term (perhaps
try "tile mapping pyglet" or variations :-)
The cocos2d engine (which works on top of pyglet) implements tile
mapping. There's probably a number of other tile mappers implemented
to run over pyglet.
If not, and your game map is entirely unique, then your texture is
still far too large. You should be able to blit your map pictures
directly to the screen and only retain those pictures in memory that
are needed for any given render.
Richard