josephwilk / rsemantic Goto Github PK

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

rsemantic's People

Contributors

Stargazers

Watchers

rsemantic's Issues

Issues with tf*idf implementation

Your implementation normalizes the frequency of a term in a document to the number of terms in that document (which, as far as I can tell, never occurs in the academic literature) and has no normalization component. My gem, on the other hand, uses the same formula as Lucene and other major implementations. See https://github.com/opennorth/tf-idf-similarity

SVD when M < N

Would it be possible to adjust LSA transform for case when M < N on document matrix like this?

module Semantic
  module Transform
    class LSA

      class << self

        def transform!(matrix, rank = nil)
          # TODO configurable rank
          columns = matrix.size2

          # if M < N perform SVD on transponsed matrix          
          matrix.size1 < matrix.size2 ? (u, v, sigma = matrix.transpose.SV_decomp_mod) : (u, v, sigma = matrix.SV_decomp_mod)

          reduce_dimensions!(sigma, rank)
          sigma = GSL::Matrix.diagonal(sigma)

          # if M < N return transposed result
          matrix.size1 < matrix.size2 ? GSL::Matrix.swap(matrix, (u * sigma * v.transpose).transpose) : GSL::Matrix.swap(matrix, u * sigma * v.transpose)
        end

Github/Gemcutter and gem releases

Any chance you could release this on gemcutter as relying on github for gem hosting doesn't make much sense anymore ?

require 'semantic'

In order to use I seem to need to :

require 'rubygems'
require 'semantic'

Shouldn't that be :

require 'rsemantic'

for consistency ?

require 'semantic' doesn't work

In order to get the gem to work I have to :


  require 'rubygems'
  require '/usr/lib/ruby/gems/1.8/gems/josephwilk-rsemantic-0.1.3/lib/semantic'

rather than :


require 'rubygems'
require 'semantic'

This just gives :


 ./semantic.rb:14: uninitialized constant Semantic (NameError)

When running :


 D1 = "The cat in the hat disabled"
 D2 = "A cat is a fine pet ponies." 
 D3 = "Dogs and cats make good pets"
 D4 = "I haven't got a hat."
 #verbose mode so it shows us all the different stages
 search = Semantic::Search.new([D1, D2, D3, D4], :verbose => true)

linalg dependancy

The linalg dependency is a bit of a nightmare for anyone running a modern gcc (4 vs 3) from what I can see. This could be avoided by supporting/using http://rb-gsl.rubyforge.org/ the ruby bindings to the gnu scientific libs.

NaN returned

Not sure if this is a bug or not, although it seems a little strange.

Running the example at : http://wiki.github.com/josephwilk/rsemantic

But changing the docs to :

documents = ["The cat in the hat disabled",
             "A cat is a fine pet ponies.",
             "Do and cats make good pets.",
             "I haven't got a cat."]

Returns :

1.0
0.0
0.0
NaN

0.0
0.0
0.0
NaN

Ruby 2.1 installation problems

Hi,
I have some problem to install rsemantic gem with ruby 2.1 (2.0 work fine) i have rb-gsl installed and gsl 1.15 lib.

gem_make.out

/home/y/.rvm/rubies/ruby-2.1.1/bin/ruby extconf.rb
checking gsl version... 1.15
checking gsl cflags... -I/usr/local/include
checking for main() in -lcblas... no
checking gsl libs... -L/usr/local/lib -lgsl -lgslcblas -lm
checking for round()... yes
checking for rngextra/rngextra.h... no
checking for qrngextra/qrngextra.h... no
checking for ool/ool_version.h... no
checking for tensor/tensor.h... no
checking for jacobi.h... no
checking for gsl/gsl_cqp.h... no
checking for gsl/gsl_multimin_fsdf.h... no
checking for gsl_poly_solve_quartic() in -lgsl... no
checking for gsl_eigen_francis() in -lgsl... yes
checking for ndlinear/gsl_multifit_ndlinear.h... no
checking for alf/alf.h... no
checking rb-gsl version...1.15.3
checking ruby version... 2.1.1
checking for graph... yes
checking for narray.h... yes
checking for tamu_anova/tamu_anova.h... no
checking for main() in -ltamuanova... no
creating Makefile

make "DESTDIR=" clean

make "DESTDIR="
compiling multimin_fsdf.c
compiling cqp.c
compiling histogram_find.c
compiling sf_expint.c
compiling const_additional.c
compiling qrng.c
compiling sf_ellint.c
compiling sf_lambert.c
compiling histogram2d.c
histogram2d.c: In function ‘mygsl_histogram2d_calloc_integrate’:
histogram2d.c:860:5: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
histogram2d.c:865:5: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
histogram2d.c:869:5: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
histogram2d.c:870:7: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
compiling eigen.c
eigen.c: In function ‘rb_gsl_eigen_genv_QZ’:
eigen.c:2075:2: warning: ‘vevec’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:2075:2: warning: ‘vbeta’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:2075:2: warning: ‘valpha’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c: In function ‘rb_gsl_eigen_genv’:
eigen.c:2024:2: warning: ‘vevec’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:2024:2: warning: ‘vbeta’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:2024:2: warning: ‘valpha’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c: In function ‘rb_gsl_eigen_gen_QZ’:
eigen.c:1910:2: warning: ‘vbeta’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:1910:2: warning: ‘valpha’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c: In function ‘rb_gsl_eigen_gen’:
eigen.c:1865:2: warning: ‘vbeta’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:1865:2: warning: ‘valpha’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c: In function ‘rb_gsl_eigen_nonsymmv_Z’:
eigen.c:1260:23: warning: ‘Z’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:1260:23: warning: ‘evec’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:1260:23: warning: ‘v’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c: In function ‘rb_gsl_eigen_nonsymmv’:
eigen.c:1125:21: warning: ‘evec’ may be used uninitialized in this function [-Wmaybe-uninitialized]
eigen.c:1125:21: warning: ‘v’ may be used uninitialized in this function [-Wmaybe-uninitialized]
compiling root.c
compiling multiset.c
compiling sf_gegenbauer.c
compiling ieee.c
compiling dirac.c
dirac.c: In function ‘rb_Dirac_matrix_whoami’:
dirac.c:341:3: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:341:3: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:341:3: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:342:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:342:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:342:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:342:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:342:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:343:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:343:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:343:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:343:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:343:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:343:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:344:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:344:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:344:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:344:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:345:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
dirac.c:345:8: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]
compiling sum.c
compiling sf_mathieu.c
compiling sf_gamma.c
compiling sf_transport.c
compiling sort.c
compiling blas2.c
compiling gsl.c
compiling vector.c
In file included from vector.c:28:0:
vector_source.c: In function ‘rb_gsl_vector_new’:
vector_source.c:286:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_get’:
vector_source.c:351:11: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
In file included from vector.c:28:0:
vector_source.c: In function ‘rb_gsl_vector_set_subvector’:
vector_source.c:415:10: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_reverse_each’:
vector_source.c:514:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_reverse_each_index’:
vector_source.c:535:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_trans_bang’:
vector_source.c:676:37: error: assignment of read-only member ‘klass’
vector_source.c:677:46: error: assignment of read-only member ‘klass’
vector_source.c: In function ‘rb_gsl_vector_connect’:
vector_source.c:760:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c:772:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_to_s’:
vector_source.c:1348:13: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_subvector_with_stride’:
vector_source.c:1412:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘mygsl_vector_shift’:
vector_source.c:1490:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘mygsl_vector_shift_scale2’:
vector_source.c:1500:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_to_gplot’:
vector_source.c:1620:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘mygsl_vector_to_m_circulant’:
vector_source.c:1702:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_delete_at’:
vector_source.c:2160:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_zip’:
vector_source.c:2874:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c:2878:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c:2885:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
In file included from vector.c:34:0:
vector_source.c: In function ‘parse_subvector_args’:
vector_source.c:98:33: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c:98:66: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_int_new’:
vector_source.c:286:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_int_get’:
vector_source.c:351:11: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
In file included from vector.c:34:0:
vector_source.c: In function ‘rb_gsl_vector_int_set_subvector’:
vector_source.c:415:10: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_int_reverse_each’:
vector_source.c:514:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_int_reverse_each_index’:
vector_source.c:535:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_int_trans_bang’:
vector_source.c:683:41: error: assignment of read-only member ‘klass’
vector_source.c:684:50: error: assignment of read-only member ‘klass’
vector_source.c: In function ‘rb_gsl_vector_int_connect’:
vector_source.c:760:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c:772:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_int_to_s’:
vector_source.c:1348:13: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_int_subvector_with_stride’:
vector_source.c:1412:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘mygsl_vector_int_shift’:
vector_source.c:1490:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘mygsl_vector_int_shift_scale2’:
vector_source.c:1500:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_int_to_gplot’:
vector_source.c:1620:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘mygsl_vector_int_to_m_circulant’:
vector_source.c:1702:3: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
vector_source.c: In function ‘rb_gsl_vector_int_delete_at’:
vector_source.c:2160:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_int_delete’:
vector_source.c:2212:20: warning: signed and unsigned type in conditional expression [-Wsign-compare]
vector_source.c: In function ‘rb_gsl_vector_int_zip’:
vector_source.c:2874:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c:2878:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
vector_source.c:2885:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
make: *** [vector.o] Error 1

make failed, exit code 2

Gems versions

Hello,

Which gems and versions do we need to run it?

Thanks!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

josephwilk / rsemantic Goto Github PK

rsemantic's Introduction

Rsemantic

Requirements:

INSTALL:

Contributors

LICENSE

rsemantic's People

Contributors

Stargazers

Watchers

Forkers

rsemantic's Issues

Recommend Projects

Recommend Topics

Recommend Org