Distributed Index Problem

[stin...@nhn.com]

I have problems regarding setting up distributed index. I've read the
documents everywhere I can find in Google.... :~~

problem 1:
ThinkingSphinx always search 127.0.0.1

in development.sphinx.conf
searchd
{
listen = xx.xx.xx.xx:3312 <= listen to specific localhost ip,
not 127.0.0.1
log = /xxxx/log/searchd.log
query_log = /xxx/log/searchd.query.log
pid_file = /xxx/log/searchd.development.pid
}

index web_core
{
source = web_core_0
path = /xxxx/db/sphinx/development/web_core
charset_type = zh_cn.utf-8
charset_dictpath = /xxxx
html_strip = 1
}

index web
{
type = distributed
agent = xx.xx.xx.xx:3312:web_core <= the IP of this server
agent = yy.yy.yy.yy:3312:web_core <= the IP of the other server
}



in controller
@results_web= Web.search @query_string, :per_page=>5


When I use browse and point to the controller url, here is the error
message

Connection to 127.0.0.1 on 3312 failed. Connection refused - connect
(2)


How can I make Thinking Sphinx and RoR to search the IP I want it to
search (xx.xx.xx.xx)






Problem2:
Ther result doesn't contain (remote distributed index) content.

With above setting, I then change searchd listen interface to any

searchd
{
listen = 3312 <= listen to all interface
log = /xxxx/log/searchd.log
query_log = /xxx/log/searchd.query.log
pid_file = /xxx/log/searchd.development.pid
}


The controller can work now.... but the result always constraint to
"local index"
Even I remark the agent, controller can still do search! But still
"local index"...NO remote agent index!!
index web
{
type = distributed
# agent = xx.xx.xx.xx:3312:web_core
# agent = yy.yy.yy.yy:3312:web_core
}


What's wrong with my config! HELP!

[Pat Allan]

To get Thinking Sphinx talking to a remote server, you need to set it
in your sphinx.yml file, as address:
http://freelancing-god.github.com/ts/en/advanced_config.html

production:
address: 10.0.0.4

Not sure about the distributed indexes, but try the above first, and
let's confirm this is working. If the distributed problem is still
happening, let me know.

--
Pat

[Sting Tao]

I've viewed the document on provided link many times. Also, I checked Sphinx documentation, too.

I just couldn't get it working as the document said...I wonder where I am wrong....

 

One question,

my ror app controller

@results_web= Web.search @query_string, :per_page=>5

The log when I did search 'rss' is:

Processing SearchController#index (for ::ffff:10.34.118.192 at 2009-11-01 02:29:38) [POST]
  Parameters: {"q"=>"rss"}
Rendering search/index
Querying Sphinx: rss
  Web Load (0.3ms)   SELECT * FROM `pages` WHERE (`pages`.`id` IN (10000015,10000023,10000024,10000025,10000026))
Completed in 19ms (View: 18, DB: 3) | 200 OK

 

I found that the default behavior of ROR is to talk to Sphinx at 127.0.0.1:3312  no matter what I wrote in config/sphinx.yml. Please teach me how to debug and what to look at?

 

 

Sting

2009/10/31 Pat Allan <p...@freelancing-gods.com>

[Pat Allan]

Hi Sting

What's the contents of your config/sphinx.yml file? Also, in script/
console, what's the output of the following line?
ThinkingSphinx::Configuration.instance.address

--
Pat

[Sting Tao]

I am still in development stage....so, I think development: part is more important...right?

 

In config/sphinx.yml

development:
mem_limit : 512MB
sql_range_step: 10000

enable_star: 1

min_prefix_len: 0

min_infix_len: 2

min_word_len: 1

max_results: 70000

morphology: none

address: 10.40.25.33
port: 3312
charset_dictpath: /env/rails_app/search_dev
stopwords: /env/rails_app/search_dev/stopwords.txt

production:
address: 10.40.25.33
port: 3312

the output of the console

# script/console
Loading development environment (Rails 2.3.4)
>> ThinkingSphinx::Configuration.instance.address
=> "127.0.0.1"
>> exit

 

 

 

 

2009/11/1 Pat Allan <p...@freelancing-gods.com>

[Pat Allan]

Hi Sting

Can you confirm that each set of values in your sphinx.yml file are
indented by at least one space? ie:
development:
mem_limit: 512M

I've just checked in the Ruby YAML parser, and this matters.

--
Pat

[Sting Tao]

This is whole configuration....

]# script/console

Loading development environment (Rails 2.3.4)
>> ThinkingSphinx::Configuration.instance

=> #<ThinkingSphinx::Configuration:0xb77a46dc @searchd_file_path="/env/rails_app/search_dev/db/sphinx/development", @searchd_binary_name="searchd", @app_root="/env/rails_app/search_dev", @index_options={:charset_type=>"utf-8"}, @delayed_job_priority=0, @database_yml_file="/env/rails_app/search_dev/config/database.yml", @configuration=#<Riddle::Configuration:0xb77a4628 @indexes=[], @indexer=#<Riddle::Configuration::Indexer:0xb77a4600 @mem_limit="512MB">, @searchd=#<Riddle::Configuration::Searchd:0xb77a45ec @pid_file="/env/rails_app/search_dev/log/searchd.development.pid", @port=3312, @log="/env/rails_app/search_dev/log/searchd.log", @address="127.0.0.1", @query_log="/env/rails_app/search_dev/log/searchd.query.log">>, @source_options={:sql_range_step=>10000}, @model_directories=["/env/rails_app/search_dev/app/models/"], @allow_star=false, @indexer_binary_name="indexer", @config_file="/env/rails_app/search_dev/config/development.sphinx.conf", @bin_path="">

2009/11/1 Sting Tao <stin...@gmail.com>

[Sting Tao]

Cool!

That solves the problem...It's really bad that I didn't check YML syntax at all. I thought it's just a simple text file.

Let me see if that solves other problems, too!

 

Pat, You are wonderful!

 

 

Sting (From Taiwan)

[Pat Allan]

Ah, good to know it's solved - hopefully the other issues aren't big
problems either :)

--
Pat

[Sting Tao]

I have correct config now...(there is one more problem that I will report later about the config....)

 

For distributed index, I found that.....

 

1. in sphinx.yml, the parameter of "address" is the address that ThinkingSphinx will talk to directly

2. in the generated config - development.sphinx.conf,  even I take away local and set agent as remote only. When my ruby app search something, only local search log showed searching activity. Remote search server didn't get any request.

 

index web
{
  type = distributed

  agent = 10.40.25.34:3312:web_core511 <= this is remote searchd
}

 

I use 0.99 gem~

 

How do I debug this? Don't know how to~~

 

Sting

 

 

2009/11/1 Pat Allan <p...@freelancing-gods.com>

[Sting Tao]

In the query log, I found that it queried the index "web_core" while my distribution index name is "web"...

How may I ask it to search the index "web"?

[Mon Nov  2 18:24:36.627 2009] 0.000 sec [all/2/rel 0 (0,5)] [web_core] HKBWS

 

2009/11/1 Sting Tao <stin...@gmail.com>

[Sting Tao]

I guess this is too basic to answer...
I read the document and add :index => "web" in my controller when I do search. This solves the problem.
Thanks.

2009/11/2 Sting Tao <stin...@gmail.com>

[Pat Allan]

Hi Sting

It's more that I was out last night and have only just woken up :)
Good to know it's all working now.

--
Pat