reading random bytes from memory

[fir]

if you will run such code (on windows)

int main()
{
int sum = 0;

for(unsigned i=0x00400000; i<0x00401000; i++)
{
char* x = (char*) i;
sum+= *x;
}

printf("sum = %d", sum);

return 0;
}
then depending on readed ram area it will work or crash ("there is a trouble with that aplication, aplication will be closed")

i think you get crash when you will read a ram area where ram is just not pinned
(im not sure if on windows pages are just guarded from read, i understand write, execute, but read? is this the case?)

that was one question second is

how to catch and recover from this crash (i just want to write a tiny ram scanner who will try read all 32 bit ram area and will give me info back which areas i can read and which i cant)

(same possibly with write though im not sure if i will try read any byte vale then write say 0x55 to any byte then write back oryginal value it will calmly stand such crash test ;c will it?

[Marcel Mueller]

On 20.08.17 23.38, fir wrote:
> for(unsigned i=0x00400000; i<0x00401000; i++)
> {
> char* x = (char*) i;
> sum+= *x;
> }

> then depending on readed ram area it will work or crash ("there is a trouble with that aplication, aplication will be closed")

Of course, this is undefined behavior.

> i think you get crash when you will read a ram area where ram is just not pinned
> (im not sure if on windows pages are just guarded from read, i understand write, execute, but read? is this the case?)

Read about virtual memory and 486 protected mode (yes 486, the model is
still valid) to get an idea about memory protection and process separation.

> that was one question second is
>
> how to catch and recover from this crash (i just want to write a tiny ram scanner who will try read all 32 bit ram area and will give me info back which areas i can read and which i cant)

Any program running at user space will never be able to read any memory
that it has not written itself before (directly or indirectly) or is
initialized by the kernel with zeros. Again read about process separation.

> (same possibly with write though im not sure if i will try read any byte vale then write say 0x55 to any byte then write back oryginal value it will calmly stand such crash test ;c will it?

If the page is not writable, it will crash.


Marcel

[Paavo Helde]

On 21.08.2017 0:38, fir wrote:
> if you will run such code (on windows)
>
> int main()
> {
> int sum = 0;
>
> for(unsigned i=0x00400000; i<0x00401000; i++)
> {
> char* x = (char*) i;
> sum+= *x;
> }
>
> printf("sum = %d", sum);
>
> return 0;
> }
> then depending on readed ram area it will work or crash ("there is a trouble with that aplication, aplication will be closed")
>
> i think you get crash when you will read a ram area where ram is just not pinned
> (im not sure if on windows pages are just guarded from read, i understand write, execute, but read? is this the case?)

The virtual memory space the process sees consists of memory pages which
are addressed indirectly through the special page tables. The problem is
not that the pages are readable, but that they are just missing (page
table entries not pointing to a normal memory page).

> that was one question second is
>
> how to catch and recover from this crash (i just want to write a tiny ram scanner who will try read all 32 bit ram area and will give me info back which areas i can read and which i cant)

In Windows and with MSVC you need to compile with /EHa option and use
catch(...) with ellipses in a C++ try block. Such a block will catch
page access violations and other stuff as an MSVC extension. In the
catch(...) block you can study the exception (by rethrowing it in the
special __try block and using GetExceptionCode() in the __except block)
to figure out if it is a normal C++ exception or a so-called "structured
exception" which covers page access errors.

Not recommended for production code, this is highly unportable and can
easily be misused for hiding program errors instead of fixing them.
Also, /EHa will make the whole program a bit slower.

If you just need to test the memory for readability or writability then
in Windows there are the IsBadReadPtr(), IsBadWritePtr() SDK functions.

Not recommended either, except for the kind of experimentation you seem
do be doing.

HTH
Paavo

[fir]

do you know maybe how its in pure c (c has signal.h header and can register some kind of handler -
im to tired now to check it myself, but would like to know if some has easy way to know that)

most ideally i would like to get full info like

"MOV instruction for IP 0x0040_110c tried to read from adress 0x0000_0012 which is page guarded from read"

and then silently continue execution
thru all those reads until my app will go thru all bytes and succesfully finish execution

[Juha Nieminen]

fir <profes...@gmail.com> wrote:
> i think you get crash when you will read a ram area where ram is just not pinned

CPU's have support for memory protection on the hardware level. This means
in practice that if a running process tries to access a memory address it's
not marked to be able to access, the CPU will throw an interrupt, which jumps
to a routine implemented by the operating system (which typically kills the
process and shows a message about the program having accessed an invalid
memory address; or something more vague in the case of Windows.)

Modern operating systems fully run in this protected mode. It stops programs
from accessing memory at will, only allowing them access to what they
explicitly allocate from the system.

On that note, likewise in modern CPUs (although it has been so for quite a
long time), the actual number in a memory address (eg. the actual value of
a pointer) has little to nothing to do with the physical location of that
memory in the RAM chips. Both the hardware and the operating system freely
remap memory addresses to physical RAM locations.

[Paavo Helde]

Anything concerning signals is not easy in my experience, doubly so with
SIGSEGV. In Windows it is 1000x easier to use IsBadReadPtr() for
studying random memory locations.

> most ideally i would like to get full info like
>
> "MOV instruction for IP 0x0040_110c tried to read from adress 0x0000_0012 which is page guarded from read"
>
> and then silently continue execution

Suggesting to look into gdb sources. Apparently it is capable of running
another program in such a fashion, with:

handle SIGSEGV nostop nopass

Just tested that, as it happens I have a handy module for that which
repeatedly reads bytes from random locations in memory. The code
originates from times when I still had the illusion that the program
could somehow survive illegal memory accesses and that this might be
useful for something. I have abandoned this illusion by now.

Below you see that gdb is printing out a line for each segfault
occurring in my program. You will need to copy that behavior from gdb
and add printouts of any additional information you are interested in.

altair: ~>gdb ac43
GNU gdb (GDB; openSUSE Leap 42.1) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.opensuse.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ac43...done.
(gdb) handle SIGSEGV nostop nopass
Signal Stop Print Pass to program Description
SIGSEGV No Yes No Segmentation fault
(gdb) run -e 'illegalmemoryaccess()'
Starting program: /users/paavo/bin/ac43 -e 'illegalmemoryaccess()'
Missing separate debuginfos, use: zypper install
glibc-debuginfo-2.19-22.1.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
process 20223 is executing new program:
/users/paavo/Acapella/trunk/Production/BuildProducts/Linux/Debug/bin/acapella
Missing separate debuginfos, use: zypper install
libgcc_s1-debuginfo-5.3.1+r233831-6.1.x86_64
libncurses5-debuginfo-5.9-53.4.x86_64 libstdc++6-debuginfo-5.3.1+r
233831-6.1.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7fffeff34700 (LWP 20232)]
[New Thread 0x7fffef533700 (LWP 20233)]
[New Thread 0x7fffee2b1700 (LWP 20234)]
[New Thread 0x7fffedab0700 (LWP 20235)]
[New Thread 0x7fffed2af700 (LWP 20236)]
[New Thread 0x7fffecaae700 (LWP 20237)]
[New Thread 0x7fffdffff700 (LWP 20238)]
[New Thread 0x7fffdf7fe700 (LWP 20239)]
[New Thread 0x7fffdeffd700 (LWP 20240)]
[New Thread 0x7fffde7fc700 (LWP 20241)]
[New Thread 0x7fffddffb700 (LWP 20242)]
[New Thread 0x7fffdd7fa700 (LWP 20243)]
[Thread 0x7fffdd7fa700 (LWP 20243) exited]
[New Thread 0x7fffdd7fa700 (LWP 20244)]
[Thread 0x7fffdd7fa700 (LWP 20244) exited]
[New Thread 0x7fffdd7fa700 (LWP 20245)]
Acapella (ver. 4.3.0.0) internal SW developer license for PerkinElmer
2017-08-22 08:40:32 1 {0/0} [acapella/debug] "IllegalMemoryAccess()
module called, process will be crashed now" |PID=20223|TID=20223|SESS=0|
"{immediate}(2) [::
IllegalMemoryAccess]"

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

Thread 1 "acapella" received signal SIGSEGV, Segmentation fault.

...

[Vir Campestris]

That's probably true of the systems that fir will meet, and it's
certainly true of anything bigger than a mobile 'phone.

It's _not_ correct for some of the smaller embedded processors and the
RTOSes that run on them - there are quite a lot of low end CPUs out
there still running in real address mode with no protection.

Andy

[Juha Nieminen]

Vir Campestris <vir.cam...@invalid.invalid> wrote:
> That's probably true of the systems that fir will meet, and it's
> certainly true of anything bigger than a mobile 'phone.

I doubt you'll find a mobile phone for which you can write your own software
which doesn't use hardward and an OS supporting (and using) memory
protection and mapping.

> It's _not_ correct for some of the smaller embedded processors and the
> RTOSes that run on them - there are quite a lot of low end CPUs out
> there still running in real address mode with no protection.

Sure, there are very simple embedded systems out there, and sometimes
you can even write software for them in C++, but from the average user's
perspective they are quite a rarity. They may be really common in certain
industries, but not amont the average programmer.

[David Brown]

Microcontrollers with linear memory (i.e., no virtual memory) totally
dominate the processor world, and vastly outnumber the devices with
virtual memory. Some of these will have memory protection units, which
can limit the access to parts of memory, but most do not have them (or
do not use them). After all, when your system only has one program,
there is no need for protection from other processes.

Many of these are programmed in C++, which has been used on small
embedded microcontrollers for decades, and has become more popular as
devices like small ARM microcontrollers have got cheaper and more
dominant. Still, they are mostly programmed in C rather than C++.

[Scott Lurndal]

David Brown <david...@hesbynett.no> writes:
>On 23/08/17 07:18, Juha Nieminen wrote:
>> Vir Campestris <vir.cam...@invalid.invalid> wrote:
>>> That's probably true of the systems that fir will meet, and it's
>>> certainly true of anything bigger than a mobile 'phone.
>>
>> I doubt you'll find a mobile phone for which you can write your own software
>> which doesn't use hardward and an OS supporting (and using) memory
>> protection and mapping.
>>
>>> It's _not_ correct for some of the smaller embedded processors and the
>>> RTOSes that run on them - there are quite a lot of low end CPUs out
>>> there still running in real address mode with no protection.
>>
>> Sure, there are very simple embedded systems out there, and sometimes
>> you can even write software for them in C++, but from the average user's
>> perspective they are quite a rarity. They may be really common in certain
>> industries, but not amont the average programmer.
>>
>
>Microcontrollers with linear memory (i.e., no virtual memory) totally
>dominate the processor world, and vastly outnumber the devices with
>virtual memory.

While that's likely true - there are, after all, still 6502's being
sold - one must consider that the actual number of programmers
who program those microcontrollers is so small it's in the noise
when contrasted with the programmers coding for more sophisticated
processors.

[David Brown]

That is a different matter, of course. But as more "mainstream"
programmers use different languages, and more people use hobby embedded
systems, I think that embedded programming is a quite a significant
proportion of C (not C++) programming.

It is extremely difficult to get any serious statistics on this sort of
thing, and my guesses could be way out.

[Vir Campestris]

On 23/08/2017 06:18, Juha Nieminen wrote:
> Vir Campestris <vir.cam...@invalid.invalid> wrote:
>> That's probably true of the systems that fir will meet, and it's
>> certainly true of anything bigger than a mobile 'phone.
>
> I doubt you'll find a mobile phone for which you can write your own software
> which doesn't use hardward and an OS supporting (and using) memory
> protection and mapping.
>

I should have worded that differently. I'm pretty sure _all_ mobiles
will have a memory controller, and definitely anything more capable.

>> It's _not_ correct for some of the smaller embedded processors and the
>> RTOSes that run on them - there are quite a lot of low end CPUs out
>> there still running in real address mode with no protection.
>
> Sure, there are very simple embedded systems out there, and sometimes
> you can even write software for them in C++, but from the average user's
> perspective they are quite a rarity. They may be really common in certain
> industries, but not amont the average programmer.
>

I suspect quite a few of the people on this group aren't average
programmers. I admit it's nearly 2 years since I used such a device.

Andy