licoy / pic-gather Goto Github PK
View Code? Open in Web Editor NEW🎨 Image collector, support for custom acquisition source, compatible with Windows and MacOS!| 图像采集器,支持自定义采集源,兼容Windows和MacOS!
License: MIT License
🎨 Image collector, support for custom acquisition source, compatible with Windows and MacOS!| 图像采集器,支持自定义采集源,兼容Windows和MacOS!
License: MIT License
增加图片队列采集进度
www.mzitu.com 这个网站也可以
2.1.0
同步采集源 提示失败
但可以下载下来 同步本地地址
Windows 10
感谢您提交您的idea,尽可能的描述清楚可以帮助我们更好的理解~
对于同样的文档 能不能跳过
gcolle
问题反馈:
1、meitulu采集过程中大约在12页左右任务会停止,开启任务后会从第一页采集。
2、部分图片未下载完,文件损坏。
软件版本:2。0.0
感谢!
mac 2.0 zip版解压后无效,dmg版本,采集过程会报错:
Uncaught Exception:
Error: Object has been destroyed
at WebContents.send (/Applications/GirlsReptile.app/Contents/Resources/electron.asar/browser/api/web-contents.js:101:15)
at /Applications/GirlsReptile.app/Contents/Resources/app.asar/dist/electron/main.js:1:39929
at /Applications/GirlsReptile.app/Contents/Resources/app.asar/node_modules/electron-download-manager/index.js:254:21
at DownloadItem.item.on (/Applications/GirlsReptile.app/Contents/Resources/app.asar/node_modules/electron-download-manager/index.js:116:21)
at emitTwo (events.js:126:13)
at DownloadItem.emit (events.js:214:7)
2.0.0
windows10
#采集源规则如下,请问是不是图片的src是相对地址的原因?
`
{
"name": "Pexels/免费素材照片",
"key":"Pexels",
"url": "https://www.pexels.com/zh-cn/",
"tags": [
{"name": "音樂", "url": "https://www.pexels.com/zh-cn/search/音樂"},
{"name": "頭髮", "url": "https://www.pexels.com/zh-cn/search/頭髮"},
{"name": "女孩", "url": "https://www.pexels.com/zh-cn/search/女孩"},
{"name": "辦公桌", "url": "https://www.pexels.com/zh-cn/search/辦公桌"},
{"name": "健身", "url": "https://www.pexels.com/zh-cn/search/健身"},
{"name": "工作", "url": "https://www.pexels.com/zh-cn/search/工作"},
{"name": "行銷", "url": "https://www.pexels.com/zh-cn/search/行銷"},
{"name": "人群", "url": "https://www.pexels.com/zh-cn/search/人群"},
{"name": "書籍", "url": "https://www.pexels.com/zh-cn/search/書籍"},
{"name": "房屋", "url": "https://www.pexels.com/zh-cn/search/房屋"},
{"name": "有創意的", "url": "https://www.pexels.com/zh-cn/search/有創意的"},
{"name": "派對", "url": "https://www.pexels.com/zh-cn/search/派對"},
{"name": "朋友", "url": "https://www.pexels.com/zh-cn/search/朋友"},
{"name": "自然", "url": "https://www.pexels.com/zh-cn/search/自然"},
{"name": "建造", "url": "https://www.pexels.com/zh-cn/search/建造"},
{"name": "肖像", "url": "https://www.pexels.com/zh-cn/search/肖像"},
{"name": "動物", "url": "https://www.pexels.com/zh-cn/search/動物"},
{"name": "建築", "url": "https://www.pexels.com/zh-cn/search/建築"},
{"name": "廚房", "url": "https://www.pexels.com/zh-cn/search/廚房"},
{"name": "質地", "url": "https://www.pexels.com/zh-cn/search/質地"},
{"name": "男性", "url": "https://www.pexels.com/zh-cn/search/男性"},
{"name": "學校", "url": "https://www.pexels.com/zh-cn/search/學校"},
{"name": "綠色", "url": "https://www.pexels.com/zh-cn/search/綠色"},
{"name": "成功", "url": "https://www.pexels.com/zh-cn/search/成功"},
{"name": "讀", "url": "https://www.pexels.com/zh-cn/search/讀"},
{"name": "樣機", "url": "https://www.pexels.com/zh-cn/search/樣機"},
{"name": "天空", "url": "https://www.pexels.com/zh-cn/search/天空"},
{"name": "健身房", "url": "https://www.pexels.com/zh-cn/search/健身房"},
{"name": "團隊", "url": "https://www.pexels.com/zh-cn/search/團隊"},
{"name": "健康", "url": "https://www.pexels.com/zh-cn/search/健康"},
{"name": "臉孔", "url": "https://www.pexels.com/zh-cn/search/臉孔"},
{"name": "研究", "url": "https://www.pexels.com/zh-cn/search/研究"},
{"name": "海", "url": "https://www.pexels.com/zh-cn/search/海"},
{"name": "時間", "url": "https://www.pexels.com/zh-cn/search/時間"},
{"name": "草", "url": "https://www.pexels.com/zh-cn/search/草"},
{"name": "抽象", "url": "https://www.pexels.com/zh-cn/search/抽象"},
{"name": "森林", "url": "https://www.pexels.com/zh-cn/search/森林"},
{"name": "會議", "url": "https://www.pexels.com/zh-cn/search/會議"},
{"name": "房間", "url": "https://www.pexels.com/zh-cn/search/房間"},
{"name": "紐約城桌面", "url": "https://www.pexels.com/zh-cn/search/紐約城桌面"},
{"name": "互聯網", "url": "https://www.pexels.com/zh-cn/search/互聯網"},
{"name": "模特兒", "url": "https://www.pexels.com/zh-cn/search/模特兒"},
{"name": "家庭成員", "url": "https://www.pexels.com/zh-cn/search/家庭成員"},
{"name": "微笑", "url": "https://www.pexels.com/zh-cn/search/微笑"},
{"name": "工作的", "url": "https://www.pexels.com/zh-cn/search/工作的"},
{"name": "市", "url": "https://www.pexels.com/zh-cn/search/市"},
{"name": "智慧手機", "url": "https://www.pexels.com/zh-cn/search/智慧手機"},
{"name": "資料", "url": "https://www.pexels.com/zh-cn/search/資料"},
{"name": "**", "url": "https://www.pexels.com/zh-cn/search/**"},
{"name": "冬季", "url": "https://www.pexels.com/zh-cn/search/冬季"},
{"name": "烹饪", "url": "https://www.pexels.com/zh-cn/search/烹饪"},
{"name": "電話", "url": "https://www.pexels.com/zh-cn/search/電話"},
{"name": "網站", "url": "https://www.pexels.com/zh-cn/search/網站"},
{"name": "地球", "url": "https://www.pexels.com/zh-cn/search/地球"},
{"name": "旅行", "url": "https://www.pexels.com/zh-cn/search/旅行"},
{"name": "工具", "url": "https://www.pexels.com/zh-cn/search/工具"},
{"name": "塗料", "url": "https://www.pexels.com/zh-cn/search/塗料"},
{"name": "有趣", "url": "https://www.pexels.com/zh-cn/search/有趣"},
{"name": "電腦", "url": "https://www.pexels.com/zh-cn/search/電腦"},
{"name": "施工", "url": "https://www.pexels.com/zh-cn/search/施工"},
{"name": "錢", "url": "https://www.pexels.com/zh-cn/search/錢"},
{"name": "攝影", "url": "https://www.pexels.com/zh-cn/search/攝影"},
{"name": "黑與白", "url": "https://www.pexels.com/zh-cn/search/黑與白"},
{"name": "家", "url": "https://www.pexels.com/zh-cn/search/家"},
{"name": "冰淇淋", "url": "https://www.pexels.com/zh-cn/search/冰淇淋"},
{"name": "禮品", "url": "https://www.pexels.com/zh-cn/search/禮品"},
{"name": "平面", "url": "https://www.pexels.com/zh-cn/search/平面"},
{"name": "爱", "url": "https://www.pexels.com/zh-cn/search/爱"},
{"name": "釀酒", "url": "https://www.pexels.com/zh-cn/search/釀酒"},
{"name": "街道", "url": "https://www.pexels.com/zh-cn/search/街道"},
{"name": "筆記本", "url": "https://www.pexels.com/zh-cn/search/筆記本"},
{"name": "非洲", "url": "https://www.pexels.com/zh-cn/search/非洲"},
{"name": "輕的", "url": "https://www.pexels.com/zh-cn/search/輕的"},
{"name": "寶寶", "url": "https://www.pexels.com/zh-cn/search/寶寶"},
{"name": "手機", "url": "https://www.pexels.com/zh-cn/search/手機"},
{"name": "紙", "url": "https://www.pexels.com/zh-cn/search/紙"},
{"name": "老", "url": "https://www.pexels.com/zh-cn/search/老"},
{"name": "山脈", "url": "https://www.pexels.com/zh-cn/search/山脈"},
{"name": "情侶", "url": "https://www.pexels.com/zh-cn/search/情侶"},
{"name": "宇宙", "url": "https://www.pexels.com/zh-cn/search/宇宙"},
{"name": "火", "url": "https://www.pexels.com/zh-cn/search/火"},
{"name": "河", "url": "https://www.pexels.com/zh-cn/search/河"},
{"name": "花卉", "url": "https://www.pexels.com/zh-cn/search/花卉"},
{"name": "假日", "url": "https://www.pexels.com/zh-cn/search/假日"},
{"name": "蘋果手機", "url": "https://www.pexels.com/zh-cn/search/蘋果手機"},
{"name": "學生", "url": "https://www.pexels.com/zh-cn/search/學生"},
{"name": "高清桌面", "url": "https://www.pexels.com/zh-cn/search/高清桌面"},
{"name": "雲", "url": "https://www.pexels.com/zh-cn/search/雲"},
{"name": "模糊", "url": "https://www.pexels.com/zh-cn/search/模糊"},
{"name": "社交媒體", "url": "https://www.pexels.com/zh-cn/search/社交媒體"},
{"name": "商業", "url": "https://www.pexels.com/zh-cn/search/商業"},
{"name": "安全", "url": "https://www.pexels.com/zh-cn/search/安全"},
{"name": "心臟", "url": "https://www.pexels.com/zh-cn/search/心臟"},
{"name": "人", "url": "https://www.pexels.com/zh-cn/search/人"},
{"name": "度假", "url": "https://www.pexels.com/zh-cn/search/度假"},
{"name": "设计", "url": "https://www.pexels.com/zh-cn/search/设计"},
{"name": "文章", "url": "https://www.pexels.com/zh-cn/search/文章"},
{"name": "地圖", "url": "https://www.pexels.com/zh-cn/search/地圖"},
{"name": "日落", "url": "https://www.pexels.com/zh-cn/search/日落"},
{"name": "雨", "url": "https://www.pexels.com/zh-cn/search/雨"},
{"name": "快樂", "url": "https://www.pexels.com/zh-cn/search/快樂"},
{"name": "汽車", "url": "https://www.pexels.com/zh-cn/search/汽車"},
{"name": "美女", "url": "https://www.pexels.com/zh-cn/search/美女"},
{"name": "蘋果", "url": "https://www.pexels.com/zh-cn/search/蘋果"},
{"name": "咖啡", "url": "https://www.pexels.com/zh-cn/search/咖啡"},
{"name": "冒險", "url": "https://www.pexels.com/zh-cn/search/冒險"},
{"name": "海灘", "url": "https://www.pexels.com/zh-cn/search/海灘"},
{"name": "晚上", "url": "https://www.pexels.com/zh-cn/search/晚上"},
{"name": "風景", "url": "https://www.pexels.com/zh-cn/search/風景"},
{"name": "女人", "url": "https://www.pexels.com/zh-cn/search/女人"},
{"name": "跳舞", "url": "https://www.pexels.com/zh-cn/search/跳舞"},
{"name": "時尚", "url": "https://www.pexels.com/zh-cn/search/時尚"},
{"name": "衣服", "url": "https://www.pexels.com/zh-cn/search/衣服"},
{"name": "樹", "url": "https://www.pexels.com/zh-cn/search/樹"},
{"name": "相機", "url": "https://www.pexels.com/zh-cn/search/相機"},
{"name": "筆電", "url": "https://www.pexels.com/zh-cn/search/筆電"},
{"name": "內部", "url": "https://www.pexels.com/zh-cn/search/內部"},
{"name": "水", "url": "https://www.pexels.com/zh-cn/search/水"},
{"name": "沙漠", "url": "https://www.pexels.com/zh-cn/search/沙漠"},
{"name": "星星", "url": "https://www.pexels.com/zh-cn/search/星星"},
{"name": "瑜伽", "url": "https://www.pexels.com/zh-cn/search/瑜伽"},
{"name": "理念", "url": "https://www.pexels.com/zh-cn/search/理念"},
{"name": "紐約", "url": "https://www.pexels.com/zh-cn/search/紐約"},
{"name": "結婚典禮", "url": "https://www.pexels.com/zh-cn/search/結婚典禮"},
{"name": "賽跑", "url": "https://www.pexels.com/zh-cn/search/賽跑"},
{"name": "太陽", "url": "https://www.pexels.com/zh-cn/search/太陽"},
{"name": "空間", "url": "https://www.pexels.com/zh-cn/search/空間"},
{"name": "年輕", "url": "https://www.pexels.com/zh-cn/search/年輕"},
{"name": "辦公室", "url": "https://www.pexels.com/zh-cn/search/辦公室"},
{"name": "工作", "url": "https://www.pexels.com/zh-cn/search/工作"},
{"name": "浪漫的", "url": "https://www.pexels.com/zh-cn/search/浪漫的"},
{"name": "自然桌面", "url": "https://www.pexels.com/zh-cn/search/自然桌面"},
{"name": "技術", "url": "https://www.pexels.com/zh-cn/search/技術"},
{"name": "化妝", "url": "https://www.pexels.com/zh-cn/search/化妝"},
{"name": "藝術", "url": "https://www.pexels.com/zh-cn/search/藝術"},
{"name": "孩子", "url": "https://www.pexels.com/zh-cn/search/孩子"},
{"name": "電腦桌面", "url": "https://www.pexels.com/zh-cn/search/電腦桌面"},
{"name": "醫", "url": "https://www.pexels.com/zh-cn/search/醫"},
{"name": "花園", "url": "https://www.pexels.com/zh-cn/search/花園"},
{"name": "人", "url": "https://www.pexels.com/zh-cn/search/人"},
{"name": "牆壁", "url": "https://www.pexels.com/zh-cn/search/牆壁"},
{"name": "金融", "url": "https://www.pexels.com/zh-cn/search/金融"},
{"name": "生意人", "url": "https://www.pexels.com/zh-cn/search/生意人"},
{"name": "放鬆", "url": "https://www.pexels.com/zh-cn/search/放鬆"},
{"name": "訓練", "url": "https://www.pexels.com/zh-cn/search/訓練"},
{"name": "行業", "url": "https://www.pexels.com/zh-cn/search/行業"},
{"name": "傷心", "url": "https://www.pexels.com/zh-cn/search/傷心"},
{"name": "夏季", "url": "https://www.pexels.com/zh-cn/search/夏季"},
{"name": "木頭", "url": "https://www.pexels.com/zh-cn/search/木頭"},
{"name": "馬路", "url": "https://www.pexels.com/zh-cn/search/馬路"},
{"name": "通訊", "url": "https://www.pexels.com/zh-cn/search/通訊"},
{"name": "餐饮", "url": "https://www.pexels.com/zh-cn/search/餐饮"},
{"name": "樹木", "url": "https://www.pexels.com/zh-cn/search/樹木"},
{"name": "组", "url": "https://www.pexels.com/zh-cn/search/组"}
],
"pages": {
"element": "body div.page-wrap div.l-container div.photos div.photos__column article",
"name": {
"element": "a:first-child img",
"attr": "alt"
},
"url": {
"element": "a:first-child",
"attr": "href",
"needMerge": true
},
"nextPageUrlMode": {
"mode": "add",
"addValue": ""
},
"lastPage": {
"element": "",
"minPageLength": 2,
"attr": "@text",
"descIndex": 2,
"isNeedSplit": false,
"splits": []
}
},
"imgs": {
"element": "div.photo-page__photo > a.js-photo-page-image-download-link",
"nextPageUrlMode": {
"incrementMode": "",
"mode": "add",
"addValue": ""
},
"lastPage": {
"element": "",
"minPageLength": 2,
"attr": "@text",
"descIndex": 2,
"isNeedSplit": false,
"splits": []
}
},
"supportReptileVersion": 1
}
`
可不可以根据用户id采集ins图片
感谢您提交您的idea,尽可能的描述清楚可以帮助我们更好的理解~
不开区分目录的情况下,如果图片重名,后面下的图片会覆盖前面的图片,不会重命名,导致下载的图片只剩后面的。
2.2.2
刚安装完启动就无法加载, 重启仍旧不能
win10
这个版本在我的联想win10电脑上安装后,打开不能加载采集源,重启也不行,多次尝试之后过来反馈
收藏源站资源加载失败
感谢您提交您的idea,尽可能的描述清楚可以帮助我们更好的理解~
新版本有个图源是需要代理的 但是我代理开了全局 pac 都不行 所以可以软件内置一下这个功能吗
希望在软件可以直接设置自己想要的采集源,那就相当完美了
本来想修改一下json文件自己试试,不过搞了很久也没成功,希望下个版本能新增这个网站支持
https://girlimg.epio.app/article
quantuwang.co/meinv/
nvshens.com
收藏源无法更新,显示加载失败
为啥源又没了,
感谢您提交您的idea,尽可能的描述清楚可以帮助我们更好的理解~
https://www.mmtp.net/ 这个网站的图片 很漂亮哦
增加对更多布局类型的网站的支持,实现对更多图片网站的下载(功能更强大,自由度更高)。
小点子:通过json来设置是否分页等,来启用不同的解析模块,以便兼容只有一层的(非内外层的)网站和无分页的网站,例如:
https://www.pexels.com
https://www.pexels.com/zh-cn/popular-searches/
...
1.用户可在界面上对采集异常进行重复采集操作
2.选择采集源后,用户可先预览目录名称,确认是否存在问题,再点击自动采集按钮
希望新增fetcherx采集
fetcherx是一个类似RSS工具,可以添加Rss/Youtube/Twitter/Instagram/Microsoft/Google Pixel/apple/Tumblr
源
比如添加某个推特源:https://app.fetcherx.com/social/twitter/NothingWe
或者是添加的Rss订阅,点击封面查看完整套图,但是在fetcherx网站内:https://app.fetcherx.com/rss/02920fe6759b55b83a10f84ce3a0a1b9
ins源:https://app.fetcherx.com/social/instagram/googlepixel
能出点cos网站的采集源嘛
彼岸图库采集,一个图片一个文件夹有点不适应
[{
“ name ”: “图片站”,//采集源站标题
“ key ”:“ key ”,//标示KEY
“ url ”: “ https://www.malefile.com/ ”,//
域名,主要用于组合链接“ tags ”: [ //分类列表
{ “ name ”: “女神”,“ url ”: “https://www.malefile.com//t/nvshen/ “ } //名称:分类名称,网址:分类地址
]
“ pages ”: { //外层页面采集规则
“ element ”: “ body> div.main> div.boxs> ul> li ”, //元素选择器
“ name ”: { //名称采集规则
“元素”: “ p.p_title> a ”, //元素选择器
“ attr ”: “ @ text ” // @text是取html元素的文本值,非此值则取attr属性,如title
},
“ url ”: { //图集采集规则
“元素”: “ p.p_title> a ”, //元素选择器
“ attr ”: “ href ”, // @ text是取html元素的文本值,非此值则取取attr属性,如标题
“ needMerge ”: false, //是否需要合并地址
},
“ nextPageUrlMode ”: { //下一页采集规则
“模式”: “添加”, //模式:添加(添加)替换(替换)
“ replaceSearchValue ”: “。 html ”, //
替换搜索字符串“ replaceValue ”: “ _ {page} .html ”, //目标
替换值“ addValue ”: “ {page} .html ” //添加值
},
“ lastPage ”: { //最后一页匹配规则,主要是用于统计总页码
“元素”: “#pages> a ”, //元素选择器
“ minPageLength ”: 2, //最小采集的到页码数量才进行匹配,否则只执行当页
“ attr ”: “ @ text ”, // @ text是取html元素的文本值,非此值则取attr属性,如标题
“ descIndex ”: 2, //页码的元素index,倒序
“ isNeedSplit ”: 不对,//是否需要分割
“ splits ”: [ //分割,例如xxx_xx_1.html的格式需要用到此值
{ “ str ”:“ _ ”, “ index ”:2 }, // str:分割字符串,索引:下次分页的字符索引
{ “ str ”:“。”, “ index ”:0 }
]
}
},
“ imgs ”: { //内页图片采集规则
“ element ”: “ body> div.content> center> img ”, //元素选择器
“ attr ”:“ href ”, // [v2]图片的属性值
“ singlePage “:true, // [v2]是否是单页面,即所有图片呈现在一个页面上
” urlNeedMerge “: true, // [v2]图片链接是否需要合并拼接
” nextPageUrlMode “:{ //同上
“ incrementMode ”: “ image ”,//采集增长模式image:图片页面:页面
“ mode ”: “ replace ”,//页面页码
替换模式,当incrementMode为图像时此值无效“ imageSuffix ”:“ jpg ” //图片后缀名
“ replaceSearchValue ”: “。 html ”,//同页此属性,当incrementMode为图像时此值无效
“ replaceValue ”: “ _ {page}。html “, //同页此属性,当incrementMode为image时此值无效
“ addValue ”: “ {page} .html ” //同页此属性,当incrementMode为image时此值无效
},
“ lastPage ”: { //通过页面此属性
“ element ”: “#pages> a ”,
“ minPageLength ”: 2,
“ attr ”: “ @ text ”,
“ descIndex ”: 2,
“ isNeedSplit ”: false,
“拆分“: []
}
},
“ supportReptileVersion ”: 1 //支持采集器的版本号
}]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.