Git Product home page Git Product logo

pywencai's Introduction

PyPI version Downloads

pywencai

获取同花顺问财数据

环境依赖

由于程序中执行了js代码,请先保证已安装了Node.js,需要版本v16+

未安装请自行安装

安装

pip install pywencai

由于问财接口策略经常发生变化,请安装最新版本使用,遇到问题时,优先尝试升级解决

视频教程

如何使用Python获取同花顺问财数据?全网最简单方法!

Demo

import pywencai

res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
print(res)

API

get(**kwargs)

根据问财语句查询结果

参数

query

必填,查询问句

老版本的question参数1.0版本以后会弃用,请以后统一使用query参数

sort_key

非必填,指定用于排序的字段,值为返回结果的列名

sort_order

非必填,排序规则,至为asc(升序)或desc(降序)

page

非必填,查询的页号,默认为1

perpage

非必填,每页数据条数,默认值100,由于问财做了数据限制,最大值为100,指定大于100的数值无效。

loop

非必填,是否循环分页,返回多页合并数据。默认值为False,可以设置为True或具体数值。

当设置为True时,程序会一直循环到最后一页,返回全部数据。

当设置具体数值n时,循环请求n页,返回n页合并数据。

query_type

非必填,默认为stock,当查询的类型不是股票的时候需要传,取值如下:

取值 含义
stock 股票
zhishu 指数
fund 基金
hkstock 港股
usstock 美股
threeboard 新三板
conbond 可转债
insurance 保险
futures 期货
lccp 理财
foreign_exchange 外汇

retry

非必填,默认为10,表示请求失败后的重试次数。

sleep

非必填,默认为0,表示循环请求时,每次请求间隔多少秒。

log

非必填,默认为False,是否在控制台打印日志。

pro

非必填,默认为False,付费版传True,

必须传入cookie参数才能使用付费版

cookie

非必填,默认为None,免费版可以忽略,付费版必须传入cookie,获取付费使用权限。

pywencai.get(question='近3个月每日市盈率', pro=True, cookie='xxxx')

cookie获取方法,复制请求头中的Cookie字段值

cookie

request_params

非必填,默认为{},可以设置额外的request参数

pywencai.get(query='昨日涨幅', sort_order='asc', loop=True, log=True, request_params={ 'proxies': proxies, 'timeout': (5, 10) })

具体参数参看:https://requests.readthedocs.io/en/latest/api/#requests.request

no_detail

非必填,默认为False,当为True时,查询一些详情类问题不再会返回字典,而返回None,可以保证查询结果类型一直为pd.DataFrameNone

find

非必填,默认为None,可以传一个数组,例如['600519', '000010'],数组内的对应标的会排列在DataFrame的最前面。

【注意】 1、该参数只有结果范围DataFrame时有效。2、配置该参数后,loop参数会失效,结果只会返回前100条。

user_agent

非必填,默认为None,可以自己传user_agent,不使用随机的生成的user_agent

返回值

当查询的是列表时,该方法返回一个pandasDataframe

当查询的是详情时,该方法返回一个字典,字典中可能包含若干个文本和Dataframe

联系方式

微信公众号

pywencai's People

Contributors

azenxu avatar flypegion avatar xmsley614 avatar zsrl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pywencai's Issues

在用conbond的时候的问题

出现 返回类型未dict
返回Key: 请填写主题
返回Key: xuangu_tableV1
返回Key: 问财选可转债
返回Key: container

数据框在 xuangu_tableV1里,但仅仅只能显示10个数据

key: 问财选可转债 q ... 股票代码
0 问题内容 ...

container {'btn1': q ... 股票代码
0 问题内容 ...

pywencai无法获取数据

一直以来可以正确使用pywencai,但突然发现无法获取数据了。比如pywencai在pypi.org的范例如下:

import pywencai

res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
print(res)

运行后出错:

Traceback (most recent call last):
  File "/Users/usr/git/div_invest/venv/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3548, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-3-93879abb35df>", line 3, in <module>
    res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/usr/git/div_invest/venv/lib/python3.11/site-packages/pywencai/wencai.py", line 185, in get
    data = params.get('data')
           ^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

AttributeError: 'NoneType' object has no attribute 'get'

File "/opt/anaconda3/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/opt/Code/BC/app.py", line 158, in home
res = pywencai.get(query='退市股票')
File "/opt/anaconda3/lib/python3.9/site-packages/pywencai/wencai.py", line 180, in get
data = params.get('data')
AttributeError: 'NoneType' object has no attribute 'get'

失效了

爬出来的数据为空,前几天还可以用。

AttributeError: 'NoneType' object has no attribute 'get'

python -V
Python 3.12.3

python test.py
Traceback (most recent call last):
File "/root/test.py", line 2, in
res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/pywencai/wencai.py", line 185, in get
data = params.get('data')
^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

问句返回结果页数过多时,loop比较影响性能。

大部分时间我们可能关注的股票是特定的某几个或者有一个确切的范围,正好问财也有相应的接口,与findDataList类似。可以传入相应参数,将自定义的股票放在结果的最前处返回。这样可以只取第一页或者前几页即可得到自己想要结果。
image

如果方便的话,我可以提个PR

pywencai.get()返回的查询结果出问题,无法正确解析为df,需要升级pywencai接口

pywencai.get()返回的查询结果无法解析为df,初步分析应该是2024.1.10日i问财官方页面更新了查询结果展示页面,导致解析查询结果数据出问题。
比如,我调用get()返回的df.columns,报错如下:AttributeError: 'dict' object has no attribute 'columns'.
经过初步分析,返回的数据类型为dict,形如:
<class 'dict'>
{'请填写主题': '

为你找到5348只符合条件的A股,具体如下:

', 'xuangu_tableV1': .......

据此分析是由于i问财官方页面更新了查询结果展示页面,导致pywencai的get接口解析html结果出错,无法生成df。不妥之处请指正。
感谢大佬们分享的pywencai,期待大佬们的版本更新,谢谢!

Get的时候报错

已经更新到了最新版本
报错如下:
/usr/local/lib/python3.8/site-packages/pywencai/hexin-v.bundle.js:102785
this[filesSymbol] ||= FileList.createImpl(this._globalObject);
^^^

SyntaxError: Unexpected token '||='
at wrapSafe (internal/modules/cjs/loader.js:979:16)
at Module._compile (internal/modules/cjs/loader.js:1027:27)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
at Module.load (internal/modules/cjs/loader.js:928:32)
at Function.Module._load (internal/modules/cjs/loader.js:769:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
at internal/main/run_main_module.js:17:47
/usr/local/lib/python3.8/site-packages/pywencai/hexin-v.bundle.js:102785
this[filesSymbol] ||= FileList.createImpl(this._globalObject);

这个是分页爬取吧?

我测试了一下,全部数据需要31秒。。。拿来做盘后分析问题不大 就是实时抓取一些需要的数据太慢了 自从问财封了接口后,短时间一次性获取所有股票的数据不方便了,能解决一下登入账号后获取全部数据的问题吗?
image

pip install pywencai报错:meson.build:2:0: ERROR: Value "c11" (of type "string") for combo option "C language standard to use" is not one of the choices.

python>pip --trusted-host pypi.org install pywencai
Collecting pywencai
Obtaining dependency information for pywencai from https://files.pythonhosted.org/packages/77/4b/28d33a6edb8d98402789cbf418063a302a96d04d563163aef07ef5f97f22/pywencai-0.12.2-py3-none-any.whl.metadata
Using cached pywencai-0.12.2-py3-none-any.whl.metadata (4.7 kB)
Collecting PyExecJS<2.0.0,>=1.5.1 (from pywencai)
Using cached PyExecJS-1.5.1.tar.gz (13 kB)
Preparing metadata (setup.py) ... done
Collecting fake-useragent<2.0.0,>=1.1.1 (from pywencai)
Obtaining dependency information for fake-useragent<2.0.0,>=1.1.1 from https://files.pythonhosted.org/packages/e4/99/60d8cf1b26938c2e0a57e232f7f15641dfcd6f8deda454d73e4145910ff6/fake_useragent-1.5.1-py3-none-any.whl.metadata
Using cached fake_useragent-1.5.1-py3-none-any.whl.metadata (15 kB)
Collecting pandas<3.0.0,>=1.5.0 (from pywencai)
Using cached pandas-2.2.2.tar.gz (4.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [13 lines of output]
+ meson setup C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766 C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766.mesonpy-1ucg6z77\build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766.mesonpy-1ucg6z77\build\meson-python-native-file.ini
The Meson build system
Version: 1.2.1
Source dir: C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766
Build dir: C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766.mesonpy-1ucg6z77\build
Build type: native build
Project name: pandas
Project version: 2.2.2
Activating VS 15.9.58

  ..\..\meson.build:2:0: ERROR: Value "c11" (of type "string") for combo option "C language standard to use" is not one of the choices. Possible choices are (as string): "none", "c89", "c99", "gnu89", "gnu90", "gnu9x", "gnu99".

  A full log can be found at C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766\.mesonpy-1ucg6z77\build\meson-logs\meson-log.txt
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip

python>node -v
v16.16.0

python>python
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:30:09) [MSC v.1934 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

最新版本无法获取数据

首先,非常感谢你的开源!
我的程序从2023年2月份一直在使用pywencai,但到2023年8月8号左右,无法返回数据。不知道是不是问财加强了反爬措施。

请问我的程序什么总是报错啊?

Traceback (most recent call last):
File "stock_wordpress_all.py", line 98, in
stock_post_creator.create_post()
File "stock_wordpress_all.py", line 60, in create_post
res_stock = pywencai.get(query=f"{self.today_wc_q} 涨停股票 剔除ST *ST股票")
File "C:\Program Files\python\lib\site-packages\pywencai\wencai.py", line 182, in get
data = params.get('data')
AttributeError: 'NoneType' object has no attribute 'get'

我 self.today_wc_q 这个数字就是当天的日期 例如20XX-XX-XX的格式。

应该更新了,可以如下方法用

res = pywencai.get(questiion='退市股票')
print(res)

注意如果之前 脚本长时间用 原来的

res = pywencai.get(query='退市股票')
print(res)

可能会被封ip一段时间

作者可否在关键位置多加点调试信息?方便排障

现在是jsdom安装了,验证node -e "require('jsdom')"成功

Jupyterlab运行, 还是一样的'NoneType' object has no attribute 'get'。

  1. JupyterLab重启过N次
  2. 用的conda环境下的nodejs,jupyter里可以正常使用nodejs。
  3. 可能jsdom安装的位置还是不对。求指教

问财官方是不是又升级了,无法爬取

[pywencai] 2023-11-22 16:12:52,412 - INFO - 获取get_robot_data失败

AttributeError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_10512\1512741554.py in

E:\DevEnvs\svn_hquant\jupyter\Global_Functions.py in h_THS_CacheStockBlock()
1962 importlib.reload(pywencai)
1963
-> 1964 res = pywencai.get(question='已省略;', sort_key='股票代码', sort_order='asc',
1965 # sleep=1,
1966 loop=True, log=True,)

E:\DevEnvs\github\pywencai\pywencai\wencai.py in get(loop, **kwargs)
180 kwargs = {replace_key(key): value for key, value in kwargs.items()}
181 params = get_robot_data(**kwargs)
--> 182 data = params.get('data')
183 condition = _.get(data, 'condition')
184 if condition is not None:

AttributeError: 'NoneType' object has no attribute 'get'

测试运行时出错

本人属小白,看到这个项目在测试时出现以下错误
C:\Users\ASUS\Scripts\python.exe D:/A股相关/test.py
Traceback (most recent call last):
File "D:\A股相关\test.py", line 2, in
res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
File "C:\Users\ASUS\lib\site-packages\pywencai\wencai.py", line 68, in get
return getPage(**kwargs)
File "C:\Users\ASUS\lib\site-packages\pywencai\wencai.py", line 32, in getPage
list = result['data']['answer'][0]['txt'][0]['content']['components'][0]['data']['datas']
KeyError: 'data'
电脑已经安装了Node.js

空指针,有排查具体问题的方法么?IP问题?

res = pywencai.get(query='多方炮')
Traceback (most recent call last):
File "", line 1, in
File "/root/anaconda3/lib/python3.11/site-packages/pywencai/wencai.py", line 182, in get
data = params.get('data')
^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

无法获取指数数据

当问题是“主力净额大于0的行业指数”时,调用pywencai的查询结果还是个股,如下图所示:
image
而在问财官网上直接提问“主力净额大于0的行业指数”时,就能得到指数的相关数据,如下图所示:
image
两者的查询结果不匹配。请问这个模块是只能查询个股吗?要怎么获取指数的相关数据呢?

调用pywencai直接报错。

Rewritten code with error handling and comments

import pywencai

try:
# Get a list of delisted stocks sorted by delisting date in ascending order
res = pywencai.get(query='市盈率<35;市净率<1.8;每股收益增长率>0;利润增长率>0;流动比率>1.8;速动比率>1;市值从小到大排序;非ST;非创业;非科创;非北证', sort_order='asc', loop=True, log=True)

# Print the result
print(res)

except Exception as e:
# Handle any exceptions that may occur
print(f"Error: {e}")

[pywencai] 2023-08-28 22:11:29,417 - INFO - 获取condition开始
[pywencai] 2023-08-28 22:11:29,496 - WARNING - 1次尝试失败
[pywencai] 2023-08-28 22:11:29,497 - WARNING - 2次尝试失败
[pywencai] 2023-08-28 22:11:29,499 - WARNING - 3次尝试失败
[pywencai] 2023-08-28 22:11:29,502 - WARNING - 4次尝试失败
[pywencai] 2023-08-28 22:11:29,505 - WARNING - 5次尝试失败
[pywencai] 2023-08-28 22:11:29,507 - WARNING - 6次尝试失败
[pywencai] 2023-08-28 22:11:29,509 - WARNING - 7次尝试失败
[pywencai] 2023-08-28 22:11:29,513 - WARNING - 8次尝试失败
[pywencai] 2023-08-28 22:11:29,514 - WARNING - 9次尝试失败
[pywencai] 2023-08-28 22:11:29,516 - WARNING - 10次尝试失败
[pywencai] 2023-08-28 22:11:29,516 - INFO - 获取get_robot_data失败
Error: 'NoneType' object has no attribute 'get'

运行报错

File "C:\Users\89731\Desktop\code\iwencai\jinri.py", line 18, in mian
page_data = pw.get(query=query_str, perpage=100, sleep=0.5,loop=True)
File "C:\ProgramData\anaconda3\lib\site-packages\pywencai\wencai.py", line 182, in get
data = params.get('data')
AttributeError: 'NoneType' object has no attribute 'get'
这个错误好玄学,刚安装那天重启下解决了,然后今天安装了几个新库,然后突然又报这个错误

问财接口更新导致列名

res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
print(res)

可以直接取get-robot-data接口里的datas数据

使用pywencai后使用logging库全局basicConfig配置不生效

问题
在全局中使用logging的basicConfig,与pywencai库冲突,全局样式不生效。
建议
建议在库中使用logger替代basicConfig来避免。

e.g.1:
import logging
import pywencai

logging.basicConfig(level=logging.INFO,
filename='work.log')
无法生成work.log

e.g.2:
case1
import logging
logging.basicConfig(level=logging.INFO)
logging.info(1)
out: [pywencai] 2023-05-15 04:16:22,364 - INFO - 1

case2
import logging
logging.basicConfig(level=logging.INFO)
logging.info(1)
out: INFO:root:1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.