소프트웨어개발/Search Engine

elasticsearch cluster 설치 + 한글형태소분석기 (CentOS 기준)

곽코딩 2012. 12. 6. 19:20
반응형

검색엔진 elasticsearch 를 설치하고, 클러스터를 구성하도록 하겠다. (CentOS기준)

우선 하나의 서버에 독립적으로 설치해보자.

(http://www.elasticsearch.org/guide/reference/setup/installation.html)


elasticsearch (검색엔진) 설치 – 한글형태소분석기 적용

 

 

다운로드 받아서 설치

# mkdir -p /data/elasticsearch

# mkdir -p /log/elasticsearch

# cd /usr/local/src

# wget https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.19.12.tar.gz

# tar zxvf elasticsearch-0.19.12.tar.gz

# cp -Rf elasticsearch-0.19.12 /usr/local/elasticsearch

# cd ../elasticsearch/

 

설정파일 수정

# vi config/elasticsearch.yml

# Cluster name identifies your cluster for auto-discovery. If you're running

# multiple clusters on the same network, make sure you're using unique names.

#

# cluster.name: elasticsearch

cluster.name: tone

 

# Path to directory where to store index data allocated for this node.

#

# path.data: /path/to/data

path.data: /data/elasticsearch

 

# Path to log files:

#

# path.logs: /path/to/logs

path.logs: /log/elasticsearch

 

# Set both 'bind_host' and 'publish_host':

#

# network.host: 192.168.0.1

network.host: 192.168.0.204

 

ES_HOME 셋팅

# vi /etc/profile

ES_HOME=/usr/local/elasticsearch

export ES_HOME

PATH=$PATH:$ES_HOME/bin:

export PATH

# source /etc/profile

 

설치가 완료되었다.

 

방화벽을 오픈하고

# iptables -I INPUT -m tcp -p tcp --dport 9200 -j ACCEPT

# iptables -I INPUT -m tcp -p tcp --dport 9300 -j ACCEPT

 

실행은 elasticsearch 명령어이며, 기본적으로 background로 실행된다.

# elasticsearch

foreground로 실행하려면

# elasticsearch -f

 

elsaticsearch를 실행하고 프로세스를 확인해보면 java를 통해 실행되는 것을 알 수 있다.

# ps -ef | grep elasticsearch

root     32477     1 50 16:15 pts/3    00:00:04 /usr/java/jdk1.6.0_37/bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.path.home=/usr/local/elasticsearch -cp :/usr/local/elasticsearch/lib/elasticsearch-0.19.12.jar:/usr/local/elasticsearch/lib/*:/usr/local/elasticsearch/lib/sigar/* org.elasticsearch.bootstrap.ElasticSearch

 

따라서, 필요하면 실행시 실행옵션을 줄 수도 있다.

# elasticsearch -Xmx1g -Xms1g -Des.index.storage.type=memory -Des.max-open-files=true

 

기능이 정상적인지 확인해보자

# curl -X GET http://192.168.0.204:9200

{

  "ok" : true,

  "status" : 200,

  "name" : "Loa",

  "version" : {

    "number" : "0.19.12",

    "snapshot_build" : false

  },

"tagline" : "You Know, for Search"

}

 

브라우저에서 상태 확인이 가능하다

http://192.168.0.204:9200/_status

http://192.168.0.204:9200/_cluster/health

http://192.168.0.204:9200/_cluster/state

 

관리툴을 설치하면 보다 편하게 브라우저에서 관리할 수 있다.

# bin/plugin -install mobz/elasticsearch-head

-> Installing mobz/elasticsearch-head...

Trying https://github.com/downloads/mobz/elasticsearch-head/elasticsearch-head-0.19.12.zip...

Trying https://github.com/mobz/elasticsearch-head/zipball/v0.19.12...

Trying https://github.com/mobz/elasticsearch-head/zipball/master...

Downloading ...................................DONE

Identified as a _site plugin, moving to _site structure ...

Installed head

 

브라우저에서 확인해보자

http://192.168.0.204:9200/_plugin/head/

 

이제는 한글형태소분석기를 설치하자

# bin/plugin -install chanil1218/elasticsearch-analysis-korean/1.1.0

-> Installing mobz/elasticsearch-head...

Trying https://github.com/downloads/mobz/elasticsearch-head/elasticsearch-head-0.19.12.zip...

Trying https://github.com/mobz/elasticsearch-head/zipball/v0.19.12...

Trying https://github.com/mobz/elasticsearch-head/zipball/master...

Downloading ...................................DONE

Identified as a _site plugin, moving to _site structure ...

Installed head

[root@localhost elasticsearch]# bin/plugin -install chanil1218/elasticsearch-analysis-korean/1.1.0

-> Installing chanil1218/elasticsearch-analysis-korean/1.1.0...

Trying https://github.com/downloads/chanil1218/elasticsearch-analysis-korean/elasticsearch-analysis-korean-1.1.0.zip...

Downloading ....................................................................................................................................................................................................................................................................................................................................DONE

Installed analysis-korean

 

elasticsearch를 재구동하고 한글형태소분석기를 설정

# curl -X PUT http://192.168.0.204:9200/test  -d '{

      "settings" : {

         "index": {

             "analysis": {

                 "analyzer": {

                     "kr_analyzer": {

                         "type": "custom"

                             , "tokenizer": "kr_tokenizer"

                             ,"filter" : ["trim","kr_filter"]

                     }

                     , "kr_analyzer": {

                         "type": "custom"

                             , "tokenizer": "kr_tokenizer"

                             ,"filter" : ["trim","kr_filter"]

                     }

                 }

             }

         }

     }

     }'

엔터를 치면

{"ok":true,"acknowledged":true}

 

elasticsearch cluster 구성

 

검색엔진 elasticsearch 를 설치하고, 한글형태소분석기까지 적용했다.

이제 클러스터를 구성하도록 하겠다.

 

192.168.0.204와 192.168.0.205 두 대의 서버에 클러스터 구성을 하겠다.

하나의 설정파일을 적용하는 걸 보여주고, 다른 서버는 node.name와 network.host 만 바꿔주면 된다.

 

192.168.0.204 의 경우

node.name=node_tone1

network.host=192.168.0.204

 

192.168.0.205 의 경우

node.name=node_tone2

network.host=192.168.0.205

 

우선, 두 대의 서버에 모두 elasticsearch와 한글형태소분석기를 설치했다고 하자.

 

192.168.0.204 서버의 설정을 하자. (대체로 숫자값은 기본값을 사용하면 된다)

cluster.name을 같게 하면 클러스터링된다.

# cd /usr/local/elasticsearch

# vi config/elasticsearch.yml

cluster.name: tone

node.name: node_tone1

node.master: true

node.date: true

node.rack: rack_tone

node.max_local_storage_nodes: 1

index.number_of_shards: 5

index.number_of_replicas: 1

index.number_of_shards: 5

index.number_of_replicas: 1

path.data: /data/elasticsearch

path.logs: /log/elasticsearch

network.host: 192.168.0.204

transport.tcp.port: 9300

http.port: 9200

gateway.type: local

gateway.recover_after_nodes: 1

gateway.recover_after_time: 5m

gateway.expected_nodes: 2

cluster.routing.allocation.node_initial_primaries_recoveries: 4

cluster.routing.allocation.node_concurrent_recoveries: 2

indices.recovery.max_size_per_sec: 0

indices.recovery.concurrent_streams: 5

discovery.zen.minimum_master_nodes: 1

discovery.zen.ping.timeout: 3s

discovery.zen.ping.multicast.enabled: true

discovery.zen.ping.unicast.hosts: ["192.168.0.204:9300", "192.168.0.205:9300"]

 

192.168.0.205 서버도 node.name와 network.host만 자기껄로 변경하자.

 

두 서버 모두 elasticsearch를 재가동

# elasticsearch

 

브라우저에서 http://192.168.0.204:9200/_plugin/head/ 나 http://192.168.0.205:9200/_plugin/head/ 를 하면

클러스터된 node_tone1 와 node_tone2가 동시에 보여진다.




저작자표시 비영리 변경금지