Comments (31)
dijs
commented on August 17, 2024
I will look into this. Thanks.
from wiki.
dijs
commented on August 17, 2024
I am not seeing any conditional requires in my code at all... @Pomax If you can dig in and find the exact line of code it cannot compile, that would be great. In the meantime, I will work on updating the babel version in hopes that a new version will compile it without bundling issues.
from wiki.
dijs
commented on August 17, 2024
I updated the compiler, please try to use the master branch for your project
from wiki.
Pomax
commented on August 17, 2024
let me give that a try
from wiki.
Pomax
commented on August 17, 2024
Hm, this is very odd. If I add a package entry for master, using:
"dependencies": {
"arctic-redpoll": "^1.0.0",
"cli-color": "^1.1.0",
"inquirer": "^1.0.2",
"md5": "^2.1.0",
"pirates": "^2.1.1",
"socket.io": "^1.4.5",
"wikijs": "https://github.com/dijs/wiki/tarball/master"
},
then a require('wikijs') yields:
C:\Users\Mike\Documents\Git projects\projects\mahjong>node
> var Wiki = require('wikijs');
Error: Cannot find module 'wikijs'
at Function.Module._resolveFilename (module.js:339:15)
at Function.Module._load (module.js:290:25)
at Module.require (module.js:367:17)
at require (internal/module.js:16:19)
at repl:1:12
at REPLServer.defaultEval (repl.js:252:27)
at bound (domain.js:287:14)
at REPLServer.runBound [as eval] (domain.js:300:12)
at REPLServer.<anonymous> (repl.js:417:12)
at emitOne (events.js:95:20)
despite the fact that there is a node_modules/wikijs dir.
from wiki.
Pomax
commented on August 17, 2024
ah hang on, I know why - master does not have a dist dir. One npm run build later and the module resolves. The webpack bundling does now throw up a ton of errors though, seemingly related to the har module
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tough-cookie/lib/cookie.js
Module not found: Error: Cannot resolve module 'net' in J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\tough-cookie\lib
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tough-cookie/lib/cookie.js 32:10-24
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tough-cookie/package.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\tough-cookie\package.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "_args": [
| [
| "tough-cookie@~2.2.0",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tough-cookie/lib/cookie.js 38:14-40
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/mime-db/db.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\mime-db\db.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "application/1d-interleaved-parityfec": {
| "source": "iana"
| },
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/mime-db/index.js 11:17-37
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/forever-agent/index.js
Module not found: Error: Cannot resolve module 'net' in J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\forever-agent
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/forever-agent/index.js 6:10-24
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/forever-agent/index.js
Module not found: Error: Cannot resolve module 'tls' in J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\forever-agent
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/forever-agent/index.js 7:10-24
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/request/lib/har.js
Module not found: Error: Cannot resolve module 'fs' in J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\request\lib
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/request/lib/har.js 3:9-22
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/cache.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\cache.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "properties": {
| "beforeRequest": {
| "$ref": "#cacheEntry"
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 4:9-32
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/cacheEntry.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\cacheEntry.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "oneOf": [{
| "type": "object",
| "optional": true,
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 5:14-42
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/content.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\content.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "size",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 6:11-36
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/cookie.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\cookie.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "name",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 7:10-34
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/creator.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\creator.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "name",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 8:11-36
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/entry.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\entry.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "optional": true,
| "required": [
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 9:9-32
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/har.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\har.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "log"
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 10:7-28
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/log.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\log.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "version",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 11:7-28
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/page.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\page.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "optional": true,
| "required": [
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 12:8-30
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/pageTimings.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\pageTimings.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "properties": {
| "onContentLoad": {
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 13:15-44
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/postData.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\postData.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "optional": true,
| "required": [
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 14:12-38
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/record.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\record.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "name",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 15:10-34
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/request.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\request.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "method",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 16:11-36
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/response.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\response.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "type": "object",
| "required": [
| "status",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 17:12-38
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/timings.json
Module parse failed: J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\har-validator\lib\schemas\timings.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "required": [
| "send",
| "wait",
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/har-validator/lib/schemas/index.js 18:11-36
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tunnel-agent/index.js
Module not found: Error: Cannot resolve module 'net' in J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\tunnel-agent
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tunnel-agent/index.js 3:10-24
ERROR in J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tunnel-agent/index.js
Module not found: Error: Cannot resolve module 'tls' in J:\Junctions\Users\Mike\Documents\Git projects\projects\mahjong\node_modules\tunnel-agent
@ J:/Junctions/Users/Mike/Documents/Git projects/projects/mahjong/~/tunnel-agent/index.js 4:10-24
from wiki.
dijs
commented on August 17, 2024
Ah right, you might have to run 'npm run build' manually in that directory
On Sat, May 21, 2016 at 11:10 AM Mike Kamermans [email protected]
wrote:
ah hang on, I know why - master does not have a dist dir.
—
You are receiving this because you commented.Reply to this email directly or view it on GitHub
#16 (comment)
from wiki.
Pomax
commented on August 17, 2024
yeah, did that, which generated the dist/wikijs file, but then requiring that in leads to the error log shown.
from wiki.
dijs
commented on August 17, 2024
Okay. Many of those errors are webpack errors related to not having loaders for those file types.
from wiki.
Pomax
commented on August 17, 2024
Hmm, that may be, but without the require('wikijs') (and just that, not even anything else) there are no webpack errors, and everything bundles perfectly fine. Do you know whether the dist/wiki.js lib still relies on lots of other things that require specific webpack loaders before things'll bundle correctly?
from wiki.
dijs
commented on August 17, 2024
These are the dependencies:
"babel-polyfill": "^6.9.0",
"isomorphic-fetch": "^2.2.1",
"underscore": "^1.8.3"Those should all be browser compatible...
from wiki.
Pomax
commented on August 17, 2024
yeah those shouldn't really be a problem... alright, I created an mcve for webpack (without babel, eslint, etc) that shows this off as minimal as I can figure.
package.json:
{
"name": "wikitest",
"version": "1.0.0",
"description": "",
"main": "test.js",
"scripts": {
"test": "webpack"
},
"author": "",
"license": "PD",
"dependencies": {
"webpack": "^1.12.9",
"wikijs": "^0.2.0"
}
}
With a simple test.js:
var Wiki = require('wikijs');
And a pretty simple webpack config:
var webpack = require('webpack');
module.exports = {
entry: "./test.js",
target: "web",
output: {
path: ".",
filename: "bundle.js"
}
};
creating these three files, running npm install and then npm test yields the same kind of errors that I was seeing in my regular project.
from wiki.
dijs
commented on August 17, 2024
Okay thanks for the test, I will do the same
On Sun, May 22, 2016 at 9:11 PM Mike Kamermans [email protected]
wrote:
yeah those shouldn't really be a problem... alright, I created an mcve for
webpack (without babel, eslint, etc) that shows this off as minimal as I
can figure.package.json:
{
"name": "wikitest",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "webpack"
},
"author": "",
"license": "PD",
"dependencies": {
"webpack": "^1.12.9",
"wikijs": "^0.2.0"
}
}With a simple test.js:
var Wiki = require('wikijs');
And a pretty simple webpack config:
var webpack = require('webpack');
module.exports = {
entry: "./test.js",
target: "web",
output: {
path: ".",
filename: "bundle.js"
}
};creating these three files, running npm install and then npm test yields
the same kind of errors that I was seeing in my regular project.—
You are receiving this because you commented.Reply to this email directly or view it on GitHub
#16 (comment)
from wiki.
dijs
commented on August 17, 2024
I should have seen this before, the issue is not with wikijs. The issue is that you are trying to import a module that cannot run on in the browser. You will have the same issue with many other server-side only modules. I may be able to make a client-side version of this library. Would that suffice?
from wiki.
Pomax
commented on August 17, 2024
that would be perfect - curious though, what does wikijs rely on that that prevents browser bundling? (Most of the things it'd need to do like net-fetching, parsing, etc. have universal packages available)
from wiki.
dijs
commented on August 17, 2024
Good question! That is actually exactly why I just started using isomorphic-fetch.
I thought that library would be fine in both browser and node, but apparently, it is not...
I guess when webpack tried to compile it, it sees request, fs, etc... and blows up...
So, maybe we should take it up with them?? Because I would love to get this working without having to create another module...
from wiki.
Pomax
commented on August 17, 2024
hm, if it's https://www.npmjs.com/package/isomorphic-fetch then it looks like it's optimized for browserify. Webpack failing on system-specific APIs makes sense, but maybe if the contract for isomorphic-fetch is identical to the browser Fetch API then it might be possible to tell webpack to treat isomorphic-fetch as an external, so that it replaces require('isomorphic-fetch') with fetch when it tries to bundle.
If that works, then I guess a simple addition to the README explaining what to do when using webpack would be the entire solution. I'll see if the three example files work with that in place.
from wiki.
Pomax
commented on August 17, 2024
Looks like even with isomorphic-fetch externalised (which itself works without a problem), something's still trying to require('request'), which causes webpack to fail.
resulting bundle:
...
var _parser = __webpack_require__(302);
...
/***/ },
/* 302 */
/***/ function(module, exports, __webpack_require__) {
var utils = __webpack_require__(303);
...
/***/ },
/* 303 */
/***/ function(module, exports, __webpack_require__) {
var request = __webpack_require__(304);
Looking at where that parser is defined in the wikijs code, it seems wiki-infobox-parser is the true culprit.
from wiki.
dijs
commented on August 17, 2024
Hmm.. do you think using either https://github.com/github/fetch or https://github.com/bitinn/node-fetch instead of https://github.com/matthew-andrews/isomorphic-fetch would fix the issue?
from wiki.
Pomax
commented on August 17, 2024
no, those won't solve the wiki-infobox-parser issue. Externalizing the fetch requirement is actually entirely possible, but that leaves the real problem of the infobox parser using require etc.
from wiki.
dijs
commented on August 17, 2024
OHH!!! I forgot about the parser.... Dang...
from wiki.
dijs
commented on August 17, 2024
I can get rid of that... My bad. I only need a small part of that library.
from wiki.
dijs
commented on August 17, 2024
Okay. Can you try test branch for me? #22
from wiki.
Pomax
commented on August 17, 2024
yup, let me test that right now
from wiki.
Pomax
commented on August 17, 2024
No bundling errors on the test files! Checking whether the codebase I was trying to use it in now also works...
from wiki.
Pomax
commented on August 17, 2024
Hm, although I do see this now:
var Wiki = require('wikijs');
var wiki = new Wiki();
leads to the following error in the browser:
utils.js:39 Uncaught TypeError: Wiki is not a constructor
(the "utils.js" seems a red herring, that's just the file that performs the console log)
from wiki.
Pomax
commented on August 17, 2024
ah, looks like the build system generates exports.default = Wiki;, but because webpack bundles the node code, things won't work with an (additional) module.exports = Wiki; (if I add that in, things work. At least, library wise. The wikipedia system of course generates a CORS error on regular use, but running it in a service worker should not be restricted by CORS)
from wiki.
dijs
commented on August 17, 2024
Excellent, so what changes do I need to make, if any?
On Wed, May 25, 2016 at 7:10 PM Mike Kamermans [email protected]
wrote:
ah, looks like the build system generates exports.default = Wiki;, but
because webpack bundles the node code, things won't work with an
(additional) module.exports = Wiki; (if I add that in, things work. At
least, library wise. The wikipedia system of course generates a CORS error
on regular use, but running it in a service worker should not be restricted
by CORS)—
You are receiving this because you commented.Reply to this email directly or view it on GitHub
#16 (comment)
from wiki.
Pomax
commented on August 17, 2024
probably just a small section in the README.md that explains that if you're using webpack, and you're experiencing problems, you probably want to use:
externals: {
"isomorphinc-fetch": "fetch"
}
Although with the parser fix that might not even be necessary anymore (not tested that and currently not in a position to quickly test)
from wiki.
dijs
commented on August 17, 2024
Thanks!
On Wed, May 25, 2016 at 11:04 PM Mike Kamermans [email protected]
wrote:
probably just a small section in the README.md that explains that if
you're using webpack, and you're experiencing problems, you probably want
to use:externals: {
"isometric-fetch": "fetch"
}—
You are receiving this because you commented.Reply to this email directly or view it on GitHub
#16 (comment)
from wiki.
Pomax
commented on August 17, 2024
and you, too!
from wiki.
Related Issues (20)
- page.title does not work HOT 1
- Page#content returns an empty array [] on Fandom Wiki HOT 1
- error TS2339: Property 'rawContent' does not exist on type 'Page'. HOT 2
- Issue viewing tables / Infoboxes HOT 2
- mainImage does not work for fr.wikipedia.org HOT 2
- mainImage fails "Invalid attempt to destructure non-iterable instance" on some specific pages HOT 2
- GeoSearch Radius is not in kilometers but in meters HOT 1
- Is it possible to chain requests? HOT 4
- wiki.search returns TypeError: wiki.search is not a function HOT 1
- How to make this chaining work? HOT 3
- [TS] Page.content() has incorrect return type HOT 3
- Include article url in most viewed list HOT 2
- Is there a way to use behind proxy?
- .info() returns wrong data HOT 2
- No way to dump templates? HOT 1
- Query support? HOT 3
- Need help: wiki() is not a function HOT 4
- Fetch smaller images for mainImage()? HOT 1
- Error when building wikijs with Vite: Cannot acces stream.Transfrom in client code
- Invalid types in ESM project
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wiki.