Comments (4)
具体报错细节为:
Traceback (most recent call last):
File "/Users/qmp/.virtualenvs/qmpspider/lib/python2.7/site-packages/apscheduler/executors/base.py", line 125, in run_job
retval = job.func(*job.args, **job.kwargs)
File "/Users/qmp/.virtualenvs/qmpspider/lib/python2.7/site-packages/mysqlsmom/mysqlsmom.py", line 321, in do_one_task
query = MyModel.raw(task["stream"]["sql"].replace("?", "%s"), (last_start_time,)).dicts().iterator()
AttributeError: 'RawQuery' object has no attribute 'iterator'
from mysqlsmom.
我的配置文件 cron_config.py
# coding=utf-8
STREAM = "CRON"
# 修改数据库连接
CONNECTION = {
'host': '127.0.0.1',
'port': 3306,
'user': 'root',
'passwd': '123456'
}
# redis存储上次同步时间等信息
REDIS = {
"host": "127.0.0.1",
"port": 6379,
"db": 0,
# "password": "password", # 不需要密码则注释或删掉该行
}
# 一次同步 BULK_SIZE 条数据到elasticsearch,不设置该配置项默认为1
BULK_SIZE = 1
# 修改elasticsearch节点
NODES = [{"host": "127.0.0.1", "port": 9200}]
TASKS = [
{
"stream": {
"database": "demo", # 在此数据库执行sql语句
"sql": "select id, name from student where update_time >= ?", # 将该sql语句选中的数据同步到 elasticsearch
"seconds": 10, # 每隔 seconds 秒同步一次,
"init_time": "2018-08-15 18:05:47" # 只有第一次同步会加载
},
"jobs": [
{
"pipeline": [
{"set_id": {"field": "id"}} # 默认设置 id字段的值 为elasticsearch中的文档id
],
"dest": {
"es": {
"action": "upsert",
"index": "demo", # 设置 index
"type": "student" # 设置 type
}
}
}
]
}
]
# CUSTOM_ROW_HANDLERS = "./my_handlers.py"
# CUSTOM_ROW_FILTERS = "./my_filters.py"
from mysqlsmom.
新建个 py27 的虚拟环境安装吧。
旧环境里 的 peewee 版本跟 mysqlsmom 用到的 peewee 版本不兼容;
from mysqlsmom.
好的大佬
from mysqlsmom.
Related Issues (20)
- 道长看我的新项目
- 如何给数据动态的设置index的,和index的名字?
- es账号密码支持配置么 HOT 2
- pip install mysqlsmom的时候找不到 HOT 1
- are you sure have example for use my_filters.py? HOT 2
- es6.6 同步有问题 HOT 3
- mysql update 同步异常 HOT 1
- python3会考虑支持吗 HOT 1
- BULK_SIZE 设置成1000 HOT 2
- 配置好,然后运行一段时间后报Killed HOT 1
- 请问支持es 7.4.0吗 HOT 1
- elasticsearch 有鉴权,如何配置 HOT 1
- mysqlsmom 线程报错 HOT 2
- mysqlsmom 手动运行 没多久就自动停了 HOT 3
- do you have an option to enable support for tls(https) elasticsearch domains? HOT 1
- 请求开发python3的版本 HOT 5
- 中文字段使用ik插件进行分词 HOT 1
- pip安装和setup.py安装的一个错误 HOT 1
- 程序不再读取binlog日志 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mysqlsmom.