Hi,
you did all the steps correctly and MapTiler generated the map exactly
at the position you have provided - without rotation, because you did
not specify it.
I guess you are not used to work with GIS tools so the easiest for you
would be to use a tool like:
http://labs.metacarta.com/rectifier/ or
http://warper.geothings.net/.
Upload your image there, mark three or more control points on the map,
warp it and download the GeoTIFF. Then you can use MapTiler on it and
there will be no troubles with position.
The alternative to those tools is mine
http://georeferencer.org/,
which will produce the World file (.wld) file for you. Here the steps
are:
1) Upload the .jpeg image to your own webserver or any free hosting on
the web
2) Type url of your image into
http://georeferencer.org/
3) Center "the cross" on your picture and on the reference map by
panning (you can choose also one of the satellite images with "+" link
on the right) and press "Add point" - this at least on three different
points on the map, more points you give more precise the result will
be
4) Press "Save & Exit" and on the next page just press "Download World
File".
5) The .wld file should have the same name like your jpeg and it
should be saved in the same directory as the jpeg file. When you open
the .jpg in MapTiler it will have already the correct georeference
including the rotation. The result will look great then.
The
georeferencer.org is now in Alpha version, we are going to rewrite
it during the next year. I plan to integrate this functionality into
the future version of MapTiler under the "Georeference" button, so it
is going to be totally user friendly - but this later on.
In case you find MapTiler useful for your maps and Garmin GPS please
consider donation via the page:
http://www.maptiler.org/garmin.html.
With a bit of support the development will go faster.
Best regards,
Petr Pridal