I always found my MySQL use many memory, even though Buffer Pool is small.
So I think Server used many memory.
I developed a demo patch (see picture), it can display how many memory used
of threads.
Expect this, I think how many sort_buffer/join_buffer/net_buffer memory
used of threads are also need display, it will let DBAs know the memory
cost detail.
> I always found my MySQL use many memory, even though Buffer Pool is small.
> So I think Server used many memory.
> I developed a demo patch (see picture), it can display how many memory
> used of threads.
> Expect this, I think how many sort_buffer/join_buffer/net_buffer memory
> used of threads are also need display, it will let DBAs know the memory
> cost detail.
The feature seems to be interesting at least to me personally. The
patch is based on 5.6 however and currently we don't have a code base
to integrate to. Feel free to open a wishlist bug in Launchpad for
Percona Server with the patch attached.
On Thu, Apr 26, 2012 at 6:57 PM, Lixun Peng <pengli...@gmail.com> wrote:
> Hi Guys,
> I always found my MySQL use many memory, even though Buffer Pool is small.
> So I think Server used many memory.
> I developed a demo patch (see picture), it can display how many memory used
> of threads.
> Expect this, I think how many sort_buffer/join_buffer/net_buffer memory used
> of threads are also need display, it will let DBAs know the memory cost
> detail.
> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to
> percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/percona-discussion?hl=en.
> The feature seems to be interesting at least to me personally. The
> patch is based on 5.6 however and currently we don't have a code base
> to integrate to. Feel free to open a wishlist bug in Launchpad for
> Percona Server with the patch attached.
> Thank you,
> Laurynas
> On Thu, Apr 26, 2012 at 6:57 PM, Lixun Peng <pengli...@gmail.com> wrote:
>> Hi Guys,
>> I always found my MySQL use many memory, even though Buffer Pool is small.
>> So I think Server used many memory.
>> I developed a demo patch (see picture), it can display how many memory used
>> of threads.
>> Expect this, I think how many sort_buffer/join_buffer/net_buffer memory used
>> of threads are also need display, it will let DBAs know the memory cost
>> detail.
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Percona Discussion" group.
>> To post to this group, send email to percona-discussion@googlegroups.com.
>> To unsubscribe from this group, send email to
>> percona-discussion+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/percona-discussion?hl=en.
> --
> You received this message because you are subscribed to the Google Groups "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/percona-discussion?hl=en.
> The feature seems to be interesting at least to me personally. The
> patch is based on 5.6 however and currently we don't have a code base
> to integrate to. Feel free to open a wishlist bug in Launchpad for
> Percona Server with the patch attached.
The patch is small and can be trivially applied to any version, even to
5.1.
But how portable this malloc_usable_size() is?
I've never heard about it before. It doesn't have a manpage (on a couple
of linux boxes that I've tried, although google shows quite a few
manpages for it). But google also shows discussion mails about removing
malloc_usable_size() from somewhere and "don't use" elsewhere.
Because I don't know how to get a pointor own how many memory, I remember
have a function "_msize" at Windows, then I found the “malloc_usable_size” at
Linux, the usage like “_msize”.
If have any other simple way to know a pointor own how many memory, I will
change this code.
> > The feature seems to be interesting at least to me personally. The
> > patch is based on 5.6 however and currently we don't have a code base
> > to integrate to. Feel free to open a wishlist bug in Launchpad for
> > Percona Server with the patch attached.
> The patch is small and can be trivially applied to any version, even to
> 5.1.
> But how portable this malloc_usable_size() is?
> I've never heard about it before. It doesn't have a manpage (on a couple
> of linux boxes that I've tried, although google shows quite a few
> manpages for it). But google also shows discussion mails about removing
> malloc_usable_size() from somewhere and "don't use" elsewhere.
> Regards,
> Sergei
> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to
> percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/percona-discussion?hl=en.
> Because I don't know how to get a pointor own how many memory, I
> remember have a function "_msize" at Windows, then I found the
> “malloc_usable_size” at Linux, the usage like “_msize”.
> If have any other simple way to know a pointor own how many memory, I
> will change this code.
I don't :)
I've only found malloc_usable_size and _msize - and only after
seeing your patch.
We (*) certainly want to track memory usage per connection and globally
in the server. I thought that the only way of doing it would be to store
the size directly in the allocated block.
But now I'll check whether malloc_usable_size/_msize are good enough.
So, I'm really interested to know how portable they are.
FreeBSD ? Mac OS X? Solaris?
I only have Linux and Mac OS X environment.
I test on GCC 4.1 and GCC 4.2, GCC 4.1 have malloc_usable_size and GCC
4.3 have malloc_size.
Which malloc_usable_size can run on Linux, and malloc_size can run on Mac
OS X.
I think we can use macro to select malloc_usable_size or malloc_size.
Before I found malloc_usable_size/ malloc_size, I tested a method that by
allocate space to store size.
Like "point=malloc(size+sizeof(size_t)); memcpy(point, &size,
sizeof(size_t)); return print+sizeof(size_t);" and "memcpy(&size, point,
sizeof(size_t)); free(point-sizeof(size_t))",
but it run test case fail, so I give up this way.
Because all of my OS are Linux, so I use "malloc_usable_size/ malloc_size "
way.
I think you can try the way above. I have no enough time to DEBUG, because
I'm a DBA, I only can try above way if I have enough time.
> > Because I don't know how to get a pointor own how many memory, I
> > remember have a function "_msize" at Windows, then I found the
> > “malloc_usable_size” at Linux, the usage like “_msize”.
> > If have any other simple way to know a pointor own how many memory, I
> > will change this code.
> I don't :)
> I've only found malloc_usable_size and _msize - and only after
> seeing your patch.
> We (*) certainly want to track memory usage per connection and globally
> in the server. I thought that the only way of doing it would be to store
> the size directly in the allocated block.
> But now I'll check whether malloc_usable_size/_msize are good enough.
> So, I'm really interested to know how portable they are.
> FreeBSD ? Mac OS X? Solaris?
> Regards,
> Sergei
> (*) We means MariaDB, not Percona :)
> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to
> percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/percona-discussion?hl=en.
> I only have Linux and Mac OS X environment. I test on GCC 4.1 and GCC
> 4.2, GCC 4.1 have malloc_usable_size and GCC 4.3 have malloc_size.
> Which malloc_usable_size can run on Linux, and malloc_size can run
> on Mac OS X.
> I think we can use macro to select malloc_usable_size or malloc_size.
Certainly.
> Before I found malloc_usable_size/ malloc_size, I tested a method
> that by allocate space to store size.
> Like "point=malloc(size+sizeof(size_t)); memcpy(point, &size,
> sizeof(size_t)); return print+sizeof(size_t);" and "memcpy(&size, point,
> sizeof(size_t)); free(point-sizeof(size_t))",
> but it run test case fail, so I give up this way.
Yes, that's what I meant. I wonder why the tests fail for you.
But never mind, it's not really important, if malloc_usable_size is
sufficient.
> Because all of my OS are Linux, so I use "malloc_usable_size/
> malloc_size " way.
> I think you can try the way above. I have no enough time to DEBUG,
> because I'm a DBA, I only can try above way if I have enough time.
Well, I think you've done more than enough already :)
I'll see if we can get it into MariaDB.
But I don't know what are Percona's criteria for including your patch.
> > I only have Linux and Mac OS X environment. I test on GCC 4.1 and GCC
> > 4.2, GCC 4.1 have malloc_usable_size and GCC 4.3 have malloc_size.
> > Which malloc_usable_size can run on Linux, and malloc_size can run
> > on Mac OS X.
> > I think we can use macro to select malloc_usable_size or malloc_size.
> Certainly.
> > Before I found malloc_usable_size/ malloc_size, I tested a method
> > that by allocate space to store size.
> > Like "point=malloc(size+sizeof(size_t)); memcpy(point, &size,
> > sizeof(size_t)); return print+sizeof(size_t);" and "memcpy(&size, point,
> > sizeof(size_t)); free(point-sizeof(size_t))",
> > but it run test case fail, so I give up this way.
> Yes, that's what I meant. I wonder why the tests fail for you.
> But never mind, it's not really important, if malloc_usable_size is
> sufficient.
> > Because all of my OS are Linux, so I use "malloc_usable_size/
> > malloc_size " way.
> > I think you can try the way above. I have no enough time to DEBUG,
> > because I'm a DBA, I only can try above way if I have enough time.
> Well, I think you've done more than enough already :)
> I'll see if we can get it into MariaDB.
> But I don't know what are Percona's criteria for including your patch.
> Regards,
> Sergei
> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to
> percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/percona-discussion?hl=en.
>> > I only have Linux and Mac OS X environment. I test on GCC 4.1 and GCC
>> > 4.2, GCC 4.1 have malloc_usable_size and GCC 4.3 have malloc_size.
>> > Which malloc_usable_size can run on Linux, and malloc_size can run
>> > on Mac OS X.
>> > I think we can use macro to select malloc_usable_size or malloc_size.
>> Certainly.
>> > Before I found malloc_usable_size/ malloc_size, I tested a method
>> > that by allocate space to store size.
>> > Like "point=malloc(size+sizeof(size_t)); memcpy(point, &size,
>> > sizeof(size_t)); return print+sizeof(size_t);" and "memcpy(&size, point,
>> > sizeof(size_t)); free(point-sizeof(size_t))",
>> > but it run test case fail, so I give up this way.
>> Yes, that's what I meant. I wonder why the tests fail for you.
>> But never mind, it's not really important, if malloc_usable_size is
>> sufficient.
>> > Because all of my OS are Linux, so I use "malloc_usable_size/
>> > malloc_size " way.
>> > I think you can try the way above. I have no enough time to DEBUG,
>> > because I'm a DBA, I only can try above way if I have enough time.
>> Well, I think you've done more than enough already :)
>> I'll see if we can get it into MariaDB.
>> But I don't know what are Percona's criteria for including your patch.
>> Regards,
>> Sergei
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Percona Discussion" group.
>> To post to this group, send email to percona-discussion@googlegroups.com.
>> To unsubscribe from this group, send email to
>> percona-discussion+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/percona-discussion?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to
> percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/percona-discussion?hl=en.
> >> On Apr 27, Lixun Peng wrote:
> >> > Hi Sergei,
> >> > I only have Linux and Mac OS X environment. I test on GCC 4.1 and GCC
> >> > 4.2, GCC 4.1 have malloc_usable_size and GCC 4.3 have malloc_size.
> >> > Which malloc_usable_size can run on Linux, and malloc_size can run
> >> > on Mac OS X.
> >> > I think we can use macro to select malloc_usable_size or
> malloc_size.
> >> Certainly.
> >> > Before I found malloc_usable_size/ malloc_size, I tested a method
> >> > that by allocate space to store size.
> >> > Like "point=malloc(size+sizeof(size_t)); memcpy(point, &size,
> >> > sizeof(size_t)); return print+sizeof(size_t);" and "memcpy(&size,
> point,
> >> > sizeof(size_t)); free(point-sizeof(size_t))",
> >> > but it run test case fail, so I give up this way.
> >> Yes, that's what I meant. I wonder why the tests fail for you.
> >> But never mind, it's not really important, if malloc_usable_size is
> >> sufficient.
> >> > Because all of my OS are Linux, so I use "malloc_usable_size/
> >> > malloc_size " way.
> >> > I think you can try the way above. I have no enough time to DEBUG,
> >> > because I'm a DBA, I only can try above way if I have enough time.
> >> Well, I think you've done more than enough already :)
> >> I'll see if we can get it into MariaDB.
> >> But I don't know what are Percona's criteria for including your patch.
> >> Regards,
> >> Sergei
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Percona Discussion" group.
> >> To post to this group, send email to
> percona-discussion@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> percona-discussion+unsubscribe@googlegroups.com.
> >> For more options, visit this group at
> >> http://groups.google.com/group/percona-discussion?hl=en.
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Percona Discussion" group.
> > To post to this group, send email to percona-discussion@googlegroups.com
> .
> > To unsubscribe from this group, send email to
> > percona-discussion+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/percona-discussion?hl=en.
> Looking for Replication with Data Consistency?
> Try Percona XtraDB Cluster!
> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to
> percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/percona-discussion?hl=en.
>> >> On Apr 27, Lixun Peng wrote:
>> >> > Hi Sergei,
>> >> > I only have Linux and Mac OS X environment. I test on GCC 4.1 and
>> >> > GCC
>> >> > 4.2, GCC 4.1 have malloc_usable_size and GCC 4.3 have
>> >> > malloc_size.
>> >> > Which malloc_usable_size can run on Linux, and malloc_size can run
>> >> > on Mac OS X.
>> >> > I think we can use macro to select malloc_usable_size or
>> >> > malloc_size.
>> >> Certainly.
>> >> > Before I found malloc_usable_size/ malloc_size, I tested a method
>> >> > that by allocate space to store size.
>> >> > Like "point=malloc(size+sizeof(size_t)); memcpy(point, &size,
>> >> > sizeof(size_t)); return print+sizeof(size_t);" and "memcpy(&size,
>> >> > point,
>> >> > sizeof(size_t)); free(point-sizeof(size_t))",
>> >> > but it run test case fail, so I give up this way.
>> >> Yes, that's what I meant. I wonder why the tests fail for you.
>> >> But never mind, it's not really important, if malloc_usable_size is
>> >> sufficient.
>> >> > Because all of my OS are Linux, so I use "malloc_usable_size/
>> >> > malloc_size " way.
>> >> > I think you can try the way above. I have no enough time to DEBUG,
>> >> > because I'm a DBA, I only can try above way if I have enough time.
>> >> Well, I think you've done more than enough already :)
>> >> I'll see if we can get it into MariaDB.
>> >> But I don't know what are Percona's criteria for including your patch.
>> >> Regards,
>> >> Sergei
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups
>> >> "Percona Discussion" group.
>> >> To post to this group, send email to
>> >> percona-discussion@googlegroups.com.
>> >> To unsubscribe from this group, send email to
>> >> percona-discussion+unsubscribe@googlegroups.com.
>> >> For more options, visit this group at
>> >> http://groups.google.com/group/percona-discussion?hl=en.
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Percona Discussion" group.
>> > To post to this group, send email to
>> > percona-discussion@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> > percona-discussion+unsubscribe@googlegroups.com.
>> > For more options, visit this group at
>> > http://groups.google.com/group/percona-discussion?hl=en.
>> Looking for Replication with Data Consistency?
>> Try Percona XtraDB Cluster!
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Percona Discussion" group.
>> To post to this group, send email to percona-discussion@googlegroups.com.
>> To unsubscribe from this group, send email to
>> percona-discussion+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/percona-discussion?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To post to this group, send email to percona-discussion@googlegroups.com.
> To unsubscribe from this group, send email to
> percona-discussion+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/percona-discussion?hl=en.
