dijs / wiki Goto Github PK

A lot of main images are labelled as logo in the wikipedia infobox. Its frustrating that there are so many labels for the same image.
I wasted a lot of time debugging it. For those who are facing the same problem please make the following change to page.js
const mainImageName = getFileName(
info.image ||
info.bildname ||
info.imagen ||
info.Immagine ||
info.badge ||
info.logo //add
);

• Use fetch https://github.com/matthew-andrews/isomorphic-fetch

Sample page: https://ru.wikipedia.org/wiki/Boletus_edulis

Hi there,

What is the best practice to handle the case when requested page does not exist?

In this case, I see following message:
UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property 'revisions' of undefined

for example:

          wiki()
            .page('asdasdlkalskdjalsdjalskdalsdjasdasd')
            .then(page => {
                page.html().then(d => console.log(d));
           })
           .catch(e => console.error(e));

Hello,
First off, thanks for the great library.

The Issue -

At the moment, the default URL query string is not compatible with all MediaWiki APIs.
The origin=* parameter causes issues on some MediaWikis.
For instance -
Running

wiki({ apiUrl: 'http://awoiaf.westeros.org/api.php' }).search('Winterfell').then(console.log)

results in this URL request -

https://awoiaf.westeros.org/api.php?list=search&srsearch=Winterfell&srlimit=50&format=json&action=query&origin=*&redirects=

Which returns an error - 'origin' parameter does not match Origin header.

Solution

Removing the origin=* parameter results in a correct API response -

https://awoiaf.westeros.org/api.php?list=search&srsearch=Winterfell&srlimit=50&format=json&action=query&redirects=

{
  "query": {
    "searchinfo": {
      "totalhits": 28
    },
    "search": [
      {
        "ns": 0,
        "title": "Winterfell",
        "snippet": "| name\t\t= <span class='searchmatch'>Winterfell</span>\n| image\t\t= [[File:Ted Nasmith A Song of Ice and Fire <span class='searchmatch'>Winterfell</span>.jpg|350px]]\n",
        "size": 28785,
        "wordcount": 4547,
        "timestamp": "2017-04-11T20:21:56Z"
      },
      {
        "ns": 0,
        "title": "Crypts beneath Winterfell",
        "snippet": "#REDIRECT [[crypt of <span class='searchmatch'>Winterfell</span>]]\n",
        "size": 55,
        "wordcount": 6,
        "timestamp": "2016-06-21T17:50:29Z"
      },
     ...
    ]
  }
}

Proposed Change

It seems that a reasonably simple solution to this would be allowing a configuration option to in the wiki() object constructor, such as -

wiki({ 
    apiUrl: 'http://awoiaf.westeros.org/api.php',
    setOriginParam: false
})

What do you think?

I'm using the plugin with webpack currently on the client, and while it behaves fine on Google Chrome it responds with an error due to CORS in Firefox.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://en.wikipedia.org/w/api.php?list=random&rnnamespace=0&rnlimit=1&format=json&action=query&redirects=&origin=*. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

No such error is thrown in Google Chrome

It would be quite helpful if, as is common with libraries using another language or something that needs to be compiled, you provided the compiled version (.js) when you released a new version.

Incidentally, thanks very much for your excellent work on this! If you can believe it, this is the only wikipedia library this fully featured, and I'm going to start using it in a big production application, which I will invite you to when we launch our public beta.

When using bluebird Promise.promisifyAll on wiki it doesn't apply on WikiPage class objects.
Like there appears method searchAsync, pageAsync but no methods of page like summaryAsync(it works if call promisification on each new object). Would be great to have WikiPage class definition exported from module(to extend once). Thanks for project.

Getting this error when calling mainImage() function.
Version used "wikijs": "^4.3.1"
Variable initialized as follow: const wiki = require('wikijs').default({ apiUrl: 'http://fr.wikipedia.org/w/api.php' }) (language fr).
Example to reproduce the problem :

   wiki.search('royan') .then(data => {
        console.log(data);
        if (data && data.results && data.results.length > 0) {
            wiki.page(data.results[0])
                .then(page => {
                    console.log(page);
                    let pageUrl = page.raw.fullurl;
                    let imageUrl = null;
                    let summary = null;
                    page.mainImage().then(img => {   //here!
                        console.log("img", img);
                        imageUrl = img;
                        page.summary().then(sum => {
                        summary = utils.getFirstSentences(sum, 3);
                });
        } else {
        }
    });

I'm using it in a project in node that already has babel-polyfill imported once, and it seems wikijs is also import it again so it's giving this error:

Error: only one instance of babel-polyfill is allowed
    at Object.<anonymous> (C:\projects\....\node_modules\wikijs\node_modules\babel-polyfill\lib\index.js:10:9)

Edit: It works for me if I remove the babel-polyfill dependency (the tests pass too, at least on node 7). I've published a package without babel-polyfill wikijs-33 in the meanwhile this is resolved, I needed it quickly in production, hope you don't mind. Awesome lib btw +1

When using the page method to get a page that doesn't exist the err variable passed to the callback function doesn't get populated. Instead I get an error saying the pages variable is not defined within wiki.min.js.

Hi,

I'm trying to use WikiJs directly from the client within webpack environment.
Unfortunately, I get the following error:

Fetch API cannot load https://en.wikipedia.org/w/api.php?prop=info%7Cpageprops&inprop=url&ppprop=disambiguation&titles=Batman&format=json&action=query&redirects=. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

which leads to:

Uncaught (in promise) TypeError: Failed to fetch

what am I missing?

Thanks in advance

trying to add this to some code that gets bundled complains that "3:15-22 require function is used in a way in which dependencies cannot be statically extracted" meaning your code's probably doing something like this:

somevar = "...";
if (...) { somevar = "......."; }
...
var Dependency = require(somevar);

instead of this:

var Dependency = false;
if (someval===val1) Dependency = require('thing1');
else if (someval===val2) Dependency = require('thing2');
else if (someval===val2) Dependency = require('thing3');

While it might look less clean, this actually makes the code far easier to both bundle and debug, as all requirements are static, and thus traceable.

• Download the Wikipedia API responses and store them as test data files.
• Use these as mock API responses alongside nock.

Hi,

First of all I have to say that, so far it's really good, exactly what I'm looking for. Well done!!..
My only problem is, that I'm getting "No article found" if the title contain some accents.. For example, if I set "Pinar del Río" as title, for which the page it is: https://en.wikipedia.org/wiki/Pinar_del_Río, I got no article found instead of the page.. I already saw that the problem, comes when requesting the page.. making the request as: "https://en.wikipedia.org/wiki/Pinar_del_R%C3%ADo", and for that the page doesn't exist.. Any idea how to fix it??

Thanks :)

First of all one simple example:

var Wiki = require('wikijs')
var wiki = new Wiki()
var when = require('when')

var searchImages = function (term) {
  var deferred = when.defer()

  wiki.page(term).then(function(page) {
    page.images().then(function(img) {
      // if wikipedia has no images, never go here
      return deferred.resolve(img)
    })
  })

  return deferred.promise
}

searchImages("potato").then(function (response) {
  console.log(response)
})

Suppose that potato has no images on wikipedia, in this case, my promise was never solve.

I had problems looking for some images at a time and when I stood waiting for the promises
to be resolved, the application stops because some Wikipedia entries doesn't have images

How can I solve this?

While using search method there could also be need for access to provided search query but this method only gives access to list of page titles (and a next() function if one limits the results). Could you please refine the search method so it also gives access to searched query?!

so it returns:
{ query: 'searched query', results: [ list of page titles ], next: [function] }

instead of:
{ results: [ list of page titles ], next: [function] }

That happens when I search for "Wayne Rooney" using https://de.wikipedia.org/w/api.php

Doesn't happen with the English Wikipedia version.

Error: Something went wrong. Could not fill variables in: (undefined) undefined
    at fillVariables (/app/node_modules/infobox-parser/build/bundle.min.js:1:1957)
    at fillVariablesUntilDone (/app/node_modules/infobox-parser/build/bundle.min.js:1:2659)
    at fillVariablesUntilDone (/app/node_modules/infobox-parser/build/bundle.min.js:1:2695)
    at getVariableValue (/app/node_modules/infobox-parser/build/bundle.min.js:1:3339)
    at reduceVariable (/app/node_modules/infobox-parser/build/bundle.min.js:1:3755)
    at /app/node_modules/infobox-parser/build/bundle.min.js:1:3935
    at Array.reduce (native)
    at extractProperties (/app/node_modules/infobox-parser/build/bundle.min.js:1:4114)
    at index (/app/node_modules/infobox-parser/build/bundle.min.js:1:9971)
    at /app/node_modules/wikijs/dist/page.js:1:1579
    at process._tickDomainCallback (internal/process/next_tick.js:135:7)

Hello,
I'm trying to get results to a certain query in 2 languages.
I know I can change the API URL but it won't help as the query returns a result on when searched for in English so looks like I need to search for the English version first and then get somehow to the page of the same entry in the desired language.

What would be the most efficient way of doing so?
I know there is a parameter called lllang in the official API but I'm not sure how to apply it through wikijs while making the request.

Thanks in advance!

Getting several errors from page.mainImage function (trying to access imageinfo on undefined). I unminified and added a check to fix for now.

By using images() I can get an array of images, but how can I get the infobox image?

Thanks for the neat library ✨

Hi, I am just wondering if there is a client side port for this project? Is this node.js only?

I was trying to build a bot that gets from wiki a simple summary about the User Search.

Everything is good, but gives a error saying Wiki isnt Constructor. Whats that?.

{ var Wiki = require('wikijs'); var term = args.join(' '); if (!term) { msg.channel.sendMessage("Please Enter a Term to Search for it in the Wiki."); return; } msg.channel.sendMessage(":mag_right: Searching for your Term..").then(m => { new Wiki().search(term, 1).then(function(data) { new Wiki().page(data.results[0]).then(function(page) { page.summary().then(function(summary) { var sumText = summary.toString().split('\n'); var continuation = function() { var paragraph = sumText.shift(); msg.channel.sendMessage(paragraph); console.log(paragraph); console.log(sumText); if (paragraph) { m.edit(paragraph); msg.channel.sendMessage(paragraph); msg.channel.sendMessage(sumText); } msg.channel.sendMessage(paragraph); }; continuation(); }); }); }, function(err) { m.edit(err); }); }) };

Thank you if you helped ^^

Attempting to query http://oldschoolrunescape.wikia.com / http://oldschoolrunescape.wikia.com/api.php

const wiki = require('wikijs').default;
wiki({
	apiUrl: 'http://oldschoolrunescape.wikia.com/api.php',
	origin: null
}).search('Bob').then(data => console.log(data));

/* (node:324) UnhandledPromiseRejectionWarning: Unhandled promise rejection (reject
ion id: 1): TypeError: Cannot read property 'search' of undefined */

Not sure what is wrong. The GoT API works fine for me.

	TypeError: Cannot read property 'imageinfo' of undefined
    at /app/node_modules/wikijs/dist/page.js:1:3865

The problem seems to be that for example in the German version (https://de.wikipedia.org/w/api.php), the prefix is not "File:" but "Datei:", in the French version it's "Fichier:". For example:

rawImages: [ { ns: 6,
    title: 'Datei:BatmobileBurton.jpg',
    missing: '',
    known: '',
    imagerepository: 'shared',
    imageinfo: [ [Object] ] },
  { ns: 6,
    title: 'Datei:Commons-logo.svg',
    missing: '',
    known: '',
    imagerepository: 'shared',
    imageinfo: [ [Object] ] },
...

Should be possible to split by indexOf(':')?

userdata = ['Narendra Modi','Salman Khan','Narendra Prasad', 'Narendra Karmarkar','Narendra Hirwani','Bill Gates', 'Elon Musk', 'Michael Jackson', 'Bob Dylan'];

 for(var i = 0; i < userdata.length; i++) {
  console.log(userdata[i]);
  wiki.page(userdata[i], function(err, page) {
    console.log(page);
  });
 }

In this case i am getting error socket hangup.If array length is less than its working fine.
Please can you give solutions for this.

The function findById returns the correct wikipedia page, but calling the function mainImage on the page returns mysteriously portugal's flag.

Output Page:

{ raw: 
   { pageid: 250197,
     ns: 0,
     title: 'France national football team',
     contentmodel: 'wikitext',
     pagelanguage: 'en',
     pagelanguagehtmlcode: 'en',
     pagelanguagedir: 'ltr',
     touched: '2017-10-16T08:45:09Z',
     lastrevid: 805578131,
     length: 90539,
     fullurl: 'https://en.wikipedia.org/wiki/France_national_football_team',
     editurl: 'https://en.wikipedia.org/w/index.php?title=France_national_football_team&action=edit',
     canonicalurl: 'https://en.wikipedia.org/wiki/France_national_football_team' },
  html: [Function: html],
  content: [Function: content],
  summary: [Function: summary],
  images: [Function: images],
  references: [Function: references],
  links: [Function: links],
  categories: [Function: categories],
  coordinates: [Function: coordinates],
  info: [Function: q],
  backlinks: [Function: backlinks],
  rawImages: [Function: h],
  mainImage: [Function: mainImage],
  langlinks: [Function: langlinks],
  rawInfo: [Function: p] }

Output MainImage:
https://upload.wikimedia.org/wikipedia/commons/5/5c/Flag_of_Portugal.svg

Code:

wiki()
	.findById(250197)
	.then(page => {
		console.log(page);
		page.mainImage().then((img) => {
			console.log(img);
		});
	});

Shows article not found error for queries like kylie jenner and does not retrieves images of some queries.

The API requests to Wikipedia are being made over HTTP. Due to this browsers are blocking the requests due to blocked:mixed-content error.

Refer https://developer.mozilla.org/en-US/docs/Web/Security/Mixed_content/How_to_fix_website_with_mixed_content

I can create a pull request if you would like to include support for HTTPS requests or for making all requests over HTTPS only.

Hi, it's me again 😄

Thanks for the parsing feature, that works great on the infobox-parser, but when I use it from wikijs, with the Dublin example, I get:

  gdp: 'US$',
  gdpPerCapita: 'US$ 51,319',

I found another one with FC Santa Coloma

Code:

wiki().page("FC Santa Coloma").then(function(page) {
  page.mainImage().then(console.log);
})

Similiar error like #53

(node:7864) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property 'imageinfo' of undefined
(node:7864) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

There is again a mismatch between title and mainImageName:

File:Santa-Coloma1.png == Image:Santa-Coloma1.png
File:Question book-new.svg == Image:Santa-Coloma1.png
File:Flag of France.svg == Image:Santa-Coloma1.png
File:Flag of Spain.svg == Image:Santa-Coloma1.png
...

Modified Code:

const image = images.find(({ title }) => {console.log(title + " == " + mainImageName);title.indexOf(mainImageName) !== -1});

The documentation link is broken. It shows a

"Cannot GET /wiki/"

error

When trying to run the example from the readme in a vanilla Node 0.10.31 environment, I get the following exception:

ReferenceError: Promise is not defined
    at Wiki.page (c:\...\node_modules\wikijs\dist\wiki.js:175:15)
    at Object.<anonymous> (c:\...\test.js:5:6)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:906:3

I don't know Babel well, but I think the build process needs to use the optional 'runtime' transformer: https://babeljs.io/docs/usage/runtime/

I modified the gulpfile to read as follows in the compile task:

            .pipe(babel({optional:['runtime']}))

I also had to install babel-runtime as a dependency of the package.

And use native JS methods where possible.

I've done the type definition. Should I make a PR to be bundled or do you want it to be part of @types? (https://github.com/DefinitelyTyped/DefinitelyTyped)

Hi,
I'd love to use wiki from Node 6.9 but since it doesn't support import I tried to use require instead.

Unfortunately, I got an error wiki is not a function.

let wiki = require('wikijs');

wiki().page('batman')
          .then(page => page.summary())
          .then(summary => {
            // some action
          })

I also tried wiki.page but got 'wiki.page' is not a function.

Does it strictly require next node version?
Thanks

Code:

var test = 3165;

wiki()
	.findById(test)
	.then(page => {
		console.log(page);
		page.mainImage().then((img) => {
			console.log(img);
		});
	});

Output:

{ raw: 
   { pageid: 3165,
     ns: 0,
     title: 'ACF Fiorentina',
     contentmodel: 'wikitext',
     pagelanguage: 'en',
     pagelanguagehtmlcode: 'en',
     pagelanguagedir: 'ltr',
     touched: '2017-10-17T20:18:19Z',
     lastrevid: 805814864,
     length: 63727,
     fullurl: 'https://en.wikipedia.org/wiki/ACF_Fiorentina',
     editurl: 'https://en.wikipedia.org/w/index.php?title=ACF_Fiorentina&action=edit',
     canonicalurl: 'https://en.wikipedia.org/wiki/ACF_Fiorentina' },
  html: [Function: html],
  content: [Function: content],
  summary: [Function: summary],
  images: [Function: images],
  references: [Function: references],
  links: [Function: links],
  categories: [Function: categories],
  coordinates: [Function: coordinates],
  info: [Function: q],
  backlinks: [Function: backlinks],
  rawImages: [Function: h],
  mainImage: [Function: mainImage],
  langlinks: [Function: langlinks],
  rawInfo: [Function: p] }
(node:7578) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Something went wrong. Could not fill variables in: (undefined) undefined

Hi, this module is still using HTTP as the base API url, so the default configuration doesn't work on Chrome since it blocks the request. I solved it by providing the apiUrl option with https.

Another issue I have, I'm getting CORS error when I try to use the module:

Fetch API cannot load https://en.wikipedia.org/w/api.php?prop=info%7Cpageprops&inprop=url&ppprop=disambiguation&titles=Batman&format=json&action=query&redirects=.
No 'Access-Control-Allow-Origin' header is present on the requested resource.
Origin 'http://localhost:3000' is therefore not allowed access.
If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

I'm using this in the browser (not in service worker), and I haven't found any possibility to pass an option for the fetch to circumvent the above issue.

I'm new to Wiki APIs, but I think that the API was recently changed, because even when I manage to get the response it can't be parsed.

Any help and/or clarification greatly appreciated, thanks!

userdata = ['accenture','Accenture top 50 business intellectuals', 'Accenture Tower'];

async.each(userdata, function(item, cb){
var peoples = [ ];
wiki.page(item, function(err, page){
page.images(function(err, data){
peoples.push({name:item,images:data});
cb();
});

  });
}, function(err, data){
   if (err) {
    return callback(err);
  }       
   return callback(null,peoples);
});

See here i am getting error "Cannot read property 'pages' of undefined" because
"Accenture top 50 business intellectuals" wiki page doesn't contain any images so its giving error if wiki page contain single image also then its working fine.
I tried to console err of page.images function also but its not giving any output and same error coming.

Please can you give me solutions of this error or if wiki page doesn't have any images then how can i solve this issues???

Thank you for your previous issues answer.Now its working fine.

The .info method does not seem to return results for all infoboxes. For example, in the article "Cross Game", the info method only returns information for the first infobox ("Infobox animanga/Header") but there are 3 infoboxes in the article (Infobox animanga/Header, Infobox animanga/Print, and Infobox animanga/Video). Is there a way to get results for all infoboxes in an article?
Code:

wiki().page('Cross Game')
    .then((page) => page.info())
    .then(console.log);

Result:

{ name: 'Cross Game',
  image: 'Cross Game v01 cover by Mitsuru Adachi.jpg',
  caption: 'Cover of the first manga volume featuring Ko (left) and Wakaba',
  jaKanji: 'クロスゲーム',
  jaRomaji: 'Kurosu Gēmu',
  genre: 'Sports' }

Expected:

[
{ name: 'Cross Game',
  image: 'Cross Game v01 cover by Mitsuru Adachi.jpg',
  caption: 'Cover of the first manga volume featuring Ko (left) and Wakaba',
  jaKanji: 'クロスゲーム',
  jaRomaji: 'Kurosu Gēmu',
  genre: 'Sports' }
,
{ type: 'manga',
  author: 'Mitsuru Adachi',
  publisher: 'Shogakukan',
  publisherEn: 'English manga publisher',
  na: 'Viz Media',
  demographic: 'Shōnen manga',
  magazine: 'Weekly Shōnen Sunday',
  first: 'May 2005',
  last: 'February 2010',
  volumes: '17',
  volumeList: 'List of Cross Game chapters' }
,
{ type: 'tv series',
  director: 'Osamu Sekita',
  studio: 'SynergySP',
  licensee: 'English anime licensee',
  na: 'Viz Media',
  network: 'TV Tokyo',
  first: 'April 5, 2009',
  last: 'March 28, 2010',
  episodes: '50',
  episodeList: 'List of Cross Game episodes' }
]

Hi there,

If an article does not have its coordinates listed as a display=title manner, this module will not locate them.

Example page:
https://en.wikipedia.org/wiki/Catanzaro

I recently added {{Coord|38|54|N|16|36|E|display=title}} to the top of the article. If you delete this line and try to find the coordinates, even though they are present in the info box, the following will result:

Error Fetching Wikipedia Coordinates for City Catanzaro because TypeError: Cannot read property '0' of undefined (My error message added, but TypeError comes from this function in page.js:

	function coordinates() {
		return (0, _util.api)(apiOptions, {
			prop: 'coordinates',
			titles: raw.title
		}).then(function (res) {
			return res.query.pages[raw.pageid].coordinates[0];
		});
	}

I'm not sure where to make adjustment to have this script search in the info box as well as the page header for coordinates. I'd be glad to make the change if someone could direct me to the relevant scripts.

Thanks!

I'm getting the same error like in Issue #51. However, I'm using the english wiki page and the page has an infobox with the File prefix.

wiki()
.page("FC Copenhagen")
.then(page => page.mainImage())
.then(console.log)
.catch(e => console.error(e));

RawImage:

 { pageid: 28199557,
    ns: 6,
    title: 'File:FC København.svg',
    imagerepository: 'local',
    imageinfo: [ [Object] ] },
... 

imageinfo:

{ url: 'https://upload.wikimedia.org/wikipedia/en/9/93/FC_K%C3%B8benhavn.svg',
 descriptionurl: 'https://en.wikipedia.org/wiki/File:FC_K%C3%B8benhavn.svg',
 descriptionshorturl: 'https://en.wikipedia.org/w/index.php?curid=28199557' }

Error:

TypeError: Cannot read property 'imageinfo' of undefined
    at node_modules/wikijs/dist/page.js:1:4213
    at process._tickCallback (internal/process/next_tick.js:109:7)

Need to be able to choose the language of the research.

By default, it does on the en.wikipedia.com, but I wanna be able to do it on fr.wikipedia and so on.

Thx

In case of the following usage

 wiki().page(data)
      .then(page => {
        console.log(page.info());
      })
      .then(console.log('Info box is parsed'))
      .catch(e => console.error(e));

in that case, I receive the following error:

TypeError: Cannot read property 'redirects' of undefined
    at a (... node_modules/wikijs/src/wiki.js:32:9)

I've installed [email protected] version

In my opinion, the site https://awoiaf.westeros.org/api.php does not produce valid content anymore.

it.only('should handle Issue #55', function() {
    this.timeout(5000);
    return wiki({
      apiUrl: 'https://awoiaf.westeros.org/api.php',
      origin: null
    })