Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Index using from tcpdump
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  12 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Creto  
View profile  
 More options Feb 14 2012, 10:51 am
From: Creto <jfcrotea...@gmail.com>
Date: Tue, 14 Feb 2012 07:51:33 -0800 (PST)
Local: Tues, Feb 14 2012 10:51 am
Subject: Index using from tcpdump
Hi everyone,
   I'm looking to get my tcpdump read by pt_index_usage, but I keep
getting an error. In fact, pt_index_usage seems to ignored my --
database parameter, and only take the latest databases on my server,
mostly like I've read in the following issue:
http://code.google.com/p/maatkit/issues/detail?id=1140

Anyone tried this before? Here is my command to capture my log:
sudo tcpdump -s 65535 -x -nn -q -tttt port 3306 | ./pt-query-digest --
type tcpdump --print --no-report > output.log

This command fills my log file pretty quickly. Then, I use the next
command to try to determine the index usage:
./pt-index-usage --host=myhost --port=3306 --user=myUser --
password=myPassword --database mySpecificDatabase output.log

When running this command, I keep having the following error:
DBD::mysql::db selectall_arrayref failed: Table
'yh_test_main.ClusterElement' doesn't exist [for Statement
"EXPLAIN ..."] at ./pt-index-usage line 4394, <> line 3.
where yh_test_main is the latest schema on my database server.

Any idea of what I'm doing wrong?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Baron Schwartz  
View profile  
 More options Feb 20 2012, 3:38 pm
From: Baron Schwartz <ba...@percona.com>
Date: Mon, 20 Feb 2012 15:38:59 -0500
Local: Mon, Feb 20 2012 3:38 pm
Subject: Re: Index using from tcpdump
It is unclear to me what you mean by "latest schema on my database server."

On Feb 14, 2012, at 10:51 AM, Creto wrote:

--
Chief Performance Architect at Percona <http://www.percona.com/>
+1 (888) 401-3401 x507
Calendar: <https://tungle.me/percona_baron>
Percona Live MySQL Conference & Expo <http://www.percona.com/live>

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Creto  
View profile  
 More options Feb 24 2012, 8:57 am
From: Creto <jfcrotea...@gmail.com>
Date: Fri, 24 Feb 2012 05:57:04 -0800 (PST)
Local: Fri, Feb 24 2012 8:57 am
Subject: Re: Index using from tcpdump
On my MySql server, I have something like 132 databases (or generally
called Schema). The one used by the tool is the one that appears last
when you sort the schemas of my servers in a alphabetic order...

On Feb 20, 3:38 pm, Baron Schwartz <ba...@percona.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Baron Schwartz  
View profile  
 More options Feb 24 2012, 9:00 am
From: Baron Schwartz <ba...@percona.com>
Date: Fri, 24 Feb 2012 09:00:23 -0500
Local: Fri, Feb 24 2012 9:00 am
Subject: Re: Index using from tcpdump
OK. Can you examine output.log and see whether the problem is in pt-query-digest or pt-index-usage?

On Feb 24, 2012, at 8:57 AM, Creto wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Creto  
View profile  
 More options Feb 24 2012, 9:13 am
From: Creto <jfcrotea...@gmail.com>
Date: Fri, 24 Feb 2012 06:13:39 -0800 (PST)
Local: Fri, Feb 24 2012 9:13 am
Subject: Re: Index using from tcpdump
Well, I tried many thing to get pt_index_usage working, but I can't
figure out what's wrong: he seems to always try to locate the tables
in the wrong schema. Right now, I made manually a "slow query log" (in
fact, it was created from a perl script). Here is a sample of it:

# User@Host: cloud_strict[cloud_strict] @ plcaronstpierre.usine.8d.com
[192.168.142.136]
# Query_time: 0.002673  Lock_time: 0.000121 Rows_sent: 10
Rows_examined: 30
use jfcroteau_bss_ftk_main;
SET timestamp=1330090066;
select this_.id as id6_0_, this_.version as version6_0_,
this_.creationDate as creation3_6_0_, this_.encryptedData as
encrypte4_6_0_, this_.name as name6_0_ from KeyStoreData this_;
# User@Host: cloud_strict[cloud_strict] @ plcaronstpierre.usine.8d.com
[192.168.142.136]
# Query_time: 0.002673  Lock_time: 0.000121 Rows_sent: 10
Rows_examined: 30
use jfcroteau_bss_ftk_main;
SET timestamp=1330090066;
select this_.id as id31_1_, this_.version as version31_1_,
this_.capStatistics_id as capStati8_31_1_, this_.definition as
definition31_1_, this_.enabled as enabled31_1_,
this_.logicalTerminal_id as logicalT9_31_1_, this_.model as
model31_1_, this_.tag as tag31_1_, this_.terminalPassphrase as
terminal7_31_1_, capstatist1_.id as id32_0_, capstatist1_.connected as
connected32_0_, capstatist1_.lastCommunication as lastComm3_32_0_,
capstatist1_.lastConnection as lastConn4_32_0_,
capstatist1_.lastConnectionFederator as lastConn5_32_0_,
capstatist1_.lastConnectionStatusChange as lastConn6_32_0_,
capstatist1_.lastDeconnection as lastDeco7_32_0_,
capstatist1_.lastKnownRemoteIpAddress as lastKnow8_32_0_ from Terminal
this_ inner join TerminalCapStatistics capstatist1_ on
this_.capStatistics_id=capstatist1_.id where
capstatist1_.lastConnectionFederator='fed001' and
capstatist1_.connected=1;

That, is 2 queries, going in tables against jfcroteau_bss_ftk_main,
let's say. Here is the command I use:
./pt-index-usage --host=myMysqlhost --port=3306 --user=user --
password=password --database jfcroteau_bss_ftk_main --create-save-
results-database --empty-save-results-tables --save-results-database
h=myMysqlhost,P=3306,u=user,p=password,D=percona
slow_query_log_emulation.log

I keep getting this error:
DBD::mysql::db selectall_arrayref failed: Table
'yh_test_main.Terminal' doesn't exist [for Statement "EXPLAIN select
myColumn from Terminal this_ inner join TerminalCapStatistics
capstatist1_ on this_.capStatistics_id=capstatist1_.id where
capstatist1_.lastConnectionFederator='something' and
capstatist1_.connected=1"] at ./pt-index-usage line 4394, <> line 2.

I don't know why, but instead of looking in my table in the
jfcroteau_bss_ftk_main, it goes against yh_test_main schema. Is it me?
Is there a problem in the file format? In the command? Is it related
to http://code.google.com/p/maatkit/issues/detail?id=1140 ?

On Feb 24, 9:00 am, Baron Schwartz <ba...@percona.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Creto  
View profile  
 More options Feb 24 2012, 10:26 am
From: Creto <jfcrotea...@gmail.com>
Date: Fri, 24 Feb 2012 07:26:32 -0800 (PST)
Local: Fri, Feb 24 2012 10:26 am
Subject: Re: Index using from tcpdump
I forgot to tell: in my perl script, I hardcoded the line "use
jfcroteau_bss_ftk_main": I thought the pt-index-usage script was using
this line to determine in which schema the table was supposed to be.

On Feb 24, 9:13 am, Creto <jfcrotea...@gmail.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Baron Schwartz  
View profile  
 More options Feb 24 2012, 10:39 am
From: Baron Schwartz <ba...@percona.com>
Date: Fri, 24 Feb 2012 10:39:50 -0500
Local: Fri, Feb 24 2012 10:39 am
Subject: Re: Index using from tcpdump
Creto,

Thanks for the additional information.  I'll try to see if I can reproduce this.  Which version of pt-index-usage are you using?

Baron

On Feb 24, 2012, at 10:26 AM, Creto wrote:

- Baron

--
Chief Performance Architect at Percona <http://www.percona.com/>
+1 (888) 401-3401 x507
Calendar: <https://tungle.me/percona_baron>
Percona Live MySQL Conference & Expo <http://www.percona.com/live>


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Baron Schwartz  
View profile  
 More options Feb 24 2012, 10:56 am
From: Baron Schwartz <ba...@percona.com>
Date: Fri, 24 Feb 2012 10:56:02 -0500
Local: Fri, Feb 24 2012 10:56 am
Subject: Re: Index using from tcpdump
Creto,

I can't reproduce this problem.  You might need to look at using PTDEBUG=1 and capture the output to a file, then inspect it to see what is happening.

Baron

On Feb 24, 2012, at 9:13 AM, Creto wrote:

- Baron

--
Chief Performance Architect at Percona <http://www.percona.com/>
+1 (888) 401-3401 x507
Calendar: <https://tungle.me/percona_baron>
Percona Live MySQL Conference & Expo <http://www.percona.com/live>


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Creto  
View profile  
 More options Feb 24 2012, 2:24 pm
From: Creto <jfcrotea...@gmail.com>
Date: Fri, 24 Feb 2012 11:24:13 -0800 (PST)
Local: Fri, Feb 24 2012 2:24 pm
Subject: Re: Index using from tcpdump
Thanks Baron for your time.

Like I've said, I tried many way to generate an input for pt-index-
usage (version 2.0.3 btw). One of them was to use tcpdump piped into
pt-query-digest then piped into pt-index-usage. Even that way (which
looks pretty straight-forward to me), as soon as I'm using a database
server which have many schemas on it, I have that kinda of error.

My command looks like this:
sudo tcpdump -s 65535 -c 1000 -x -nn -q -tttt port 3306 | ./pt-query-
digest --type tcpdump --print --no-report | ./pt-index-usage.8d --
host=myMysqlHost --port=3306 --user=user --password=password --
database jfcroteau_bss_ftk_main --create-save-results-database --empty-
save-results-tables --save-results-database
h=myMsqlHost,P=3306,u=user,p=password,D=percona

Same thing trying to use pt-index-usage with our mysql slow query log.
It really seems that pt-index-usage can't deal with a database server
having more than 1 schema on it. Or I don't use the tool correctly?

Thank you.

On Feb 24, 10:56 am, Baron Schwartz <ba...@percona.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Baron Schwartz  
View profile  
 More options Feb 24 2012, 2:46 pm
From: Baron Schwartz <ba...@percona.com>
Date: Fri, 24 Feb 2012 14:46:47 -0500
Local: Fri, Feb 24 2012 2:46 pm
Subject: Re: Index using from tcpdump
Creto,

I'm pretty sure you've found a bug.  Can you do the following and see what happens?

1. Use your short sample file that you showed before.
2. Run the following command.

PTDEBUG=1 ./pt-index-usage --
host=myMysqlHost --port=3306 --user=user --password=password --
database jfcroteau_bss_ftk_main --create-save-results-database --empty-
save-results-tables --save-results-database
h=myMsqlHost,P=3306,u=user,p=password,D=percona > out.txt 2>&1

Examine the out.txt file and see if you can figure out what it's doing wrong.

- Baron


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Creto  
View profile  
 More options Feb 27 2012, 11:54 am
From: Creto <jfcrotea...@gmail.com>
Date: Mon, 27 Feb 2012 08:54:23 -0800 (PST)
Local: Mon, Feb 27 2012 11:54 am
Subject: Re: Index using from tcpdump
Hi Baron,
   I'm back on this problem. I'm try to figure out what is wrong with
the sample. Can I send you files privately? Because those files
contains sensitive data.

Quick question: the "use someScheme;" statement in the slow query log
is not always present. Is it normal? Is the algorith is something like
all the queries after a "use" are all against database until the next
use? I'm only asking for debugging purpose. Can't read the
confirmation of it on the web.

Thank you.

On Feb 24, 2:46 pm, Baron Schwartz <ba...@percona.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Baron Schwartz  
View profile  
 More options Feb 27 2012, 1:54 pm
From: Baron Schwartz <ba...@percona.com>
Date: Mon, 27 Feb 2012 13:54:35 -0500
Local: Mon, Feb 27 2012 1:54 pm
Subject: Re: Index using from tcpdump
Hi!

Yes, you can send me the files privately.  MySQL prints a "use"
statement every time the current database changes.  We mimic this in
pt-query-digest. Unfortunately there is no way to do "use NULL" when a
statement doesn't have a default database.

On 02/27/2012 11:54 AM, Creto wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »