guyaross / openweb.net Goto Github PK

OpenWeb .Net is a Library that provides high-level access to finding answers to your questions on the web. That includes functionality for fact checking, as well a finding answers to more general questions.

Extracted for public use from Project Orva. Expansive of the Summarizer .Net project.

Import all of the OpenWeb Dlls and external dependencies from the distribute directory into your .net project solution.

• Summarizer.Core
• Summarizer.Infrastructure
• Newtonsoft.Json
• HtmlAgilityPack
• OpenNLP

Open Web utilities OpenNlp, to use, extract the NLP contents within the project resources directory and place it within a nlp directory within your project's bin.

Utilizing the web for finding questions requires an instance of ISearchDomain. As well as an instance of IKeywordExtractor from the Summarizer .Net project.

Additionally, the web search invocation returns a dictionary of ranked answers utilizing a variant of the Page Rank Algorithm.

using Summarizer.Core.KeywordExtractors;

using OpenWeb.Core;

// ...

string question; // some question

ISearchDomain searchDomain; // set search domain instance
IKeywordExtractor keywordExtractor; // set keyword extractor instance

var webSearch = new WebSearch(searchDomain, keywordExtractor);

Dictionary<string, double> rankedAnswers = webSearch.Invoke(question); // ranked keywords

OpenWeb provides high level extensions for the ranked answers returned from the web search invocation.

using Summarizer.Core.KeywordExtractors;

using OpenWeb.Core;

// ...
Dictionary<string, double> rankedAnswers = webSearch.Invoke(question); // ranked keywords

Dictionary<string, double> orderedAnswers = rankedAnswers.OrderByHighest();

using Summarizer.Core.KeywordExtractors;

using OpenWeb.Core;

// ...
Dictionary<string, double> rankedAnswers = webSearch.Invoke(question); // ranked keywords

Dictionary<string, double> orderedAnswers = rankedAnswers.OrderByLowest();

using Summarizer.Core.KeywordExtractors;

using OpenWeb.Core;

// ...
Dictionary<string, double> rankedAnswers = webSearch.Invoke(question); // ranked keywords

string topAnswer = rankedAnswers.SelectHead();

The web search class comes preloaded with a set of tuned settings, providing both accuracy and performance impacts.

using Summarizer.Core.KeywordExtractors;

using OpenWeb.Core;

// ...
ISearchDomain searchDomain;
IKeywordExtractor keywordExtractor;

SearchSettingsType settings;

var webSearch = new WebSearch(searchDomain, keywordExtractor, settings);

This number sets the amount of data links cycled through at run-time. Which subsequently affects the size of the overall data-set used to determine the correct answer as these links are based off of what the queried search domain returns.

Default Value: 2

This number sets the amount of max paragraphs ever generated from any one data link source. Which similarly to the max length size, will have a direct impact over the size of the data-set used to determine the correct answers.

Default Value: 3

using Summarizer.Core.KeywordExtractors;

using OpenWeb.Core;

// ...
SearchSettingsType settings;
settings.MaxLinks = 3; // max amount of links set
settings.MaxParagraphs = 5; // max amount of paragraphs

Implications of the comparison includes fact checking an already defined answer against the answer that would be provided from the web search.

using Summarizer.Core.KeywordExtractors;

using OpenWeb.Core;

// ...
Dictionary<string, double> rankedAnswers = webSearch.Invoke(question); // ranked keywords

IEnumerable<string> keywords; // keyword data set to compare against
Dictionary<string, double> orderedAnswers = rankedAnswers.ScoreFromKeyword(keywords);

Any instance of ISearchDomain dictates from what search engine the initial question query will originate from.

• GoogleSearchDomain

Feel free to contribute by opening a Pull Request or an issue thread.

Contributions are always appreciated!

OpenWeb .Net is MIT licensed