简要咨询咨询QQ网站导航网站搜索手机站点联系我们设为首页加入收藏 

Elasticsearch中使用reiver-jdbc导入数据

来源:易贤网   阅读:2900 次  日期:2014-11-03 12:12:16

温馨提示:易贤网小编为您整理了“Elasticsearch中使用reiver-jdbc导入数据”,方便广大网友查阅!

Elastisearch中提供了river模块来从其他数据源中获取数据,该项功能以插件的形式存在,目前已有的river插件包括:

river pluginsedit

1. Supported by Elasticsearch

CouchDB River Plugin

RabbitMQ River Plugin

Twitter River Plugin

Wikipedia River Plugin

2. Supported by the community

ActiveMQ River Plugin (by Dominik Dorn)

Amazon SQS River Plugin (by Alex Bogdanovski)

CSV River Plugin (by Martin Bednar)

Dropbox River Plugin (by David Pilato)

FileSystem River Plugin (by David Pilato)

Git River Plugin (by Olivier Bazoud)

GitHub River Plugin (by uberVU)

Hazelcast River Plugin (by Steve Samuel)

JDBC River Plugin (by J?rg Prante)

JMS River Plugin (by Steve Sarandos)

Kafka River Plugin (by Endgame Inc.)

LDAP River Plugin (by Tanguy Leroux)

MongoDB River Plugin (by Richard Louapre)

Neo4j River Plugin (by Steve Samuel)

Open Archives Initiative (OAI) River Plugin (by J?rg Prante)

Redis River Plugin (by Steve Samuel)

RSS River Plugin (by David Pilato)

Sofa River Plugin (by adamlofts)

Solr River Plugin (by Luca Cavanna)

St9 River Plugin (by Sunny Gleason)

Subversion River Plugin (by Pascal Lombard)

DynamoDB River Plugin (by Kevin Wang)

可以看出,已经覆盖了大部分的数据源,特别是针对关系型数据库提供了统一的jdbc-river来进行数据操作。

elasticsearch-river-jdbc的源码在:github.com/jprante/elasticsearch-river-jdbc,该项目提供了详细的文档,下面以SQL Server为例简单说明使用方法。

首先,需要安装elasticsearch-river-jdbc,在elasticsearch目录下执行:

./bin/plugin --install jdbc --url xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-river-jdbc/1.1.0.1/elasticsearch-river-jdbc-1.1.0.1-plugin.zip

然后,安装SQLServer的JDBC库,链接为: Microsoft JDBC Driver.把其中的 'sqljdbc4.jar'复制到elasticsearch安装目录的lib文件夹下。

考虑到elasticsearch集群,以上两个步骤在每个节点上都需要执行。

最后也是最关键的一步,在elasticsearch中建立river,让elasticsearch自动从SQLServer中获取数据。

PUT /_river/mytest_river/_meta

{

"type" : "jdbc",

"jdbc" : {

"driver":"com.microsoft.sqlserver.jdbc.SQLServerDriver",

"url":"jdbc:sqlserver://MYSQLSERVERNAME;databaseName=MYProductDatabase",

"user":"admin","password":"Password",

"sql":"select ProductID as _id, CategoryID,ManufacturerID,MfName,ProductTitle,MfgPartNumber from MyProductsTable(nolock)",

"poll":"10m",

"strategy" : "simple",

"index" : "myinventory",

"type" : "product",

"bulk_size" : 100,

"max_retries": 5,

"max_retries_wait":"30s",

"max_bulk_requests" : 5,

"bulk_flush_interval" : "5s"

}

}

更多信息请查看IT技术专栏

更多信息请查看数据库
点此处就本文及相关问题在本站进行非正式的简要咨询(便捷快速)】     【点此处查询各地各类考试咨询QQ号码及交流群
上一篇:单用户模式:"数据库需要处于单用户模式下"怎么解决
下一篇:利用MSSQL复制功能实现与Oracle数据库同步
易贤网手机网站地址:Elasticsearch中使用reiver-jdbc导入数据
由于各方面情况的不断调整与变化,易贤网提供的所有考试信息和咨询回复仅供参考,敬请考生以权威部门公布的正式信息和咨询为准!