I found the solution for this problem. None of suggestions that i found works for me, so i tried using this steps:
1: rename the file modelsim-pe_student_edition.exe to modelsim-pe_student_edition.rar
2: unpacking this file and change the properties of Setup.exe and _ISDel.exe. The only configuration that i had success was changing the "compatible configuration for all users" setting the compatible mode to Windows98/Me and giving administrator permissions.
3: On Control Panel, click Program and Resources and select the Internet Explorer 11.
4: double click on Setup.exe and installation is started. This may take few minutes, but for me this was the only way to install.
5: At the end of the installation process, select Finish and a browser window will open with the License Request form
ModelSim PE Student Edition Info:
**** Please note - clicking on an existing license request link from your browser bookmark or from a link posted on the web - WILL NOT WORK.