Git Product home page Git Product logo

redmine_xapian's Introduction

Redmine Xapian search plugin

The current version of Redmine Xapian is 1.6.6
With this plugin you will be able to make searches by file name and by strings inside your documents through the Xapian search engine.
This plugin can also index the files located in your repositories.
This plugin replaces search controller, its view and search methods.

1. Installation and Setup

A copy of the plugin can be downloaded from Github at https://github.com/xelkano/redmine_xapian/downloads

1.1. Required packages

Redmine >= 3.0

To use xapian search engine you must install libxapian-ruby and xapian-omega packages (on Ubuntu). To index some files with omega you may have to install some other packages like xpdf, antiword, …

From Omega documentation:

  • PDF (.pdf) if pdftotext is available (comes with xpdf)
  • PostScript (.ps, .eps, .ai) if ps2pdf (from ghostscript) and pdftotext (comes with xpdf) are available
  • OpenOffice/StarOffice documents (.sxc, .stc, .sxd, .std, .sxi, .sti, .sxm, .sxw, .sxg, .stw) if unzip is available
  • OpenDocument format documents (.odt, .ods, .odp, .odg, .odc, .odf, .odb, .odi, .odm, .ott, .ots, .otp, .otg, .otc, .otf, .oti, .oth) if unzip is available
  • MS Word documents (.doc, .dot) if antiword is available
  • MS Excel documents (.xls, .xlb, .xlt) if xls2csv is available (comes with catdoc)
  • MS Powerpoint documents (.ppt, .pps) if catppt is available (comes with catdoc)
  • MS Office 2007 documents (.docx, .dotx, .xlsx, .xlst, .pptx, .potx, .ppsx) if unzip is available
  • Wordperfect documents (.wpd) if wpd2text is available (comes with libwpd)
  • MS Works documents (.wps, .wpt) if wps2text is available (comes with libwps)
  • AbiWord documents (.abw)
  • Compressed AbiWord documents (.zabw) if gzip is available
  • Rich Text Format documents (.rtf) if unrtf is available
  • Perl POD documentation (.pl, .pm, .pod) if pod2text is available
  • TeX DVI files (.dvi) if catdvi is available
  • DjVu files (.djv, .djvu) if djvutxt is available
  • XPS files (.xps) if unzip is available

On Debian you can use following command to install required indexing tools:

# sudo apt-get install xapian-omega libxapian-dev xpdf poppler-utils antiword unzip \
catdoc libwpd-tools libwps-tools gzip unrtf catdvi djview djview3 uuid uuid-dev

1.2. Plugin installation

Install redmine_xapian into the plugins directory with:

# cd redmine/plugins
# git clone https://github.com/xelkano/redmine_xapian.git
# cd ..
# bundle install
# bundle exec rake redmine:plugins:migrate RAILS_ENV="production"

And after that restart the application server.

Now, you can see new check boxes “Files” and “Repositories” on search screen, those allows you to search attachments by file name and its contents.
Xapian plugin checks for ruby bindings before its startup. If the plugin can not find them, it is not activated and the following message is going to appear in Redmine log “REDMAIN_XAPIAN ERROR: No Ruby bindings for Xapian installed !!. PLEASE install Xapian search engine interface for Ruby" If you see this message, please make sure that Xapian library are correctly installed for your ruby environment.

1.3. Setup

First at all, go to the Redmine interface and in the plugins section configure the plugin. It’s very important to set up correctly the directory that will contain the Xapian databases.

To use stemmed languages in searches, Xapian needs to generate stemmed forms for each language and store them in a database.
Before you can use the plugin, you have to populate the Xapian databases for each stemmed language you want to use. From plugin version 1.6 a new script (xapian_indexer.rb) is included in the extra directory that is use to populate Xapian databases. This script is a wrapper for omindex binary, so omindex should not to be used anymore.

First edit the script file and configure the required variables, they are self-explanatory and most of them can be specified and overwritten through the command line.

To view command line help simply run “xapian_index.rb -h”. To view the complete process you can run it using verbose flag (-v).

Note, that the first time you index a repository it can take a long time. After the first indexation the process is much faster.
You have to take into account that the script needs a defined repository identifier for all repositories that will be indexed. The repository identifier is not required in redmine and can have null value for some of the repositories, so before indexing a repository you have to be sure that the repository has a defined identifier, otherwise the script will ignore it.

There is a rake task for setting up a default identifier for your repositories:

bundle exec rake redmine:xapian_set_identifiers identifier='main' RAILS_ENV="production"

Examples running the script:

Run the script based in its configuration in verbose mode, index repositories and files:

# ruby xapian_indexer.rb -v

Only index redmine files:

# ruby xapian_indexer.rb -v -f

Only index repositories:

# xapian_indexer.rb -v -r

Indexing repositories of some projects:

# xapian_indexer.rb -v -r -p project_identifier1,project_identifier2

Configure a cron task for automatic indexing

Once you have tested the script functionality, you can configure it for running every day at nights keeping your Xapian database up to date, for example:

@daily www-data ruby /usr/bin/xapian_indexer.rb

After files indexing and the plugin setup in the settings you can search attachments by name and content.

To perform searches in a particular module such as Files, Documents,… it is
mandatory to enable these modules in your projects and to have corresponding
permissions view_*; Be sure it is checked.

There is a one hook allowing plugins to implement a quick jump to an object:

:controller_search_quick_jump

redmine_xapian's People

Contributors

xelkano avatar picman avatar nettsundere avatar cforce avatar akzhan avatar duanejevon avatar duboisst avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.