Git Product home page Git Product logo

contentaggregators's Introduction

说明

特色:
1.前端纯HTML+JS+JSON(链接统计除外),后端python生成标准JSON
2.自带30几个常用网站采集规则
3.多线程抓取,30+网站5秒内采集完毕
4.相同网站放在一个框架内,可局部刷新和滚动
5.更新时间显示(按采集页的自带更新时间或按时间排序的第一条时间采集,没有的为采集完成时间)
6.内容链接防盗链,链接通过base64+字符逆序+大小写反转+base64实现加密(更换大小写转换顺序或增加数字替换可实现不同密码加密)
7.内容链接点击统计
8.部分内容鼠标悬停提示(如appstore排行有更新内容、时间、版本号)

演示:
https://make.mk
https://www.make.mk (cdn镜像站)

安装:
1.python3 caiji.py采集(保持后台运行请用screen或nohup)
2.打开首页访问

注:
caiji.py为后端采集程序,其余为前端需在同一目录
相应设置修改请查看源码中的注释
admin.php为第三方数据库管理程序,用于内容链接统计查看(默认密码admin,请在文件中修改)
点击统计数据库log.db会在首次访问内容链接时生成,如未建立请保证主目录可写
为了安全,后端程序caiji.py和数据库查看程序admin.php请改为复杂文件名,或移动到非web目录(需同时修改文件内的生成json保存目录为网站根目录,默认为当前目录)
默认使用多线程抓取,如有问题请改用单线程
今日头条的采集请更换为自己的cookie,头条短时间采集内容相同的太多(目前1分钟采一次合并10次数据),相同cookie分段采集后几乎不会有重复的,但采集一段时间后推荐的都是杂七杂八无关的垃圾内容(请定期更换cookie或自行加入随机的头条cookie生成规则)
采集站的logo大部分已修改为透明图层png,在非白色背景下可能会很难看

contentaggregators's People

Contributors

asiacny avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.