asmuth / recommendify Goto Github PK
View Code? Open in Web Editor NEWGenerate recommendations using collaborative filtering
Generate recommendations using collaborative filtering
Right now if you're trying to work from a Redis database other than 0, you must use the Ruby similarity implementation, because the database info does not get passed to the C extension.
I don't understand how cosine works. It looks like it does not implement the correct functions.
input_matrix :thing_other,
# :native => true,
:similarity_func => :cosine
recommender.thing_other.process!
Traceback (most recent call last):
1: from (irb):13
RuntimeError (implemented in subclass)
Looking at the code, it looks like there are unimplemented methods on the class. Is this expected?
Is it right to recommend only p(roduct)4, p5 for "for('p3')" when I input add_set("u1", ["p1","p2","p3"]); add_set("u2", ["p3", "p4", "p5"])?
I think it is some kind of bug in algorithm, because I think it is equivalent p1, p2, p4, and p5.
Am I wrong?
Creating an input matrix (jaccard) with a redis db other than 0 results in the correct items and ccmatrix being created as data is added, but empty similarities values when process! is called. This appears to be because the db is not taken into account by the native recommendify command. AFAICT it never gets passed to it.
I am trying to install and its impossible to me. I have installed hiredis and it is in my directory: /usr/local/include
how is the correct way? Here is the context of what i have made: http://stackoverflow.com/questions/20594338/unable-to-install-recommendify-gem-hiredis-h-not-found
Thanks in advance
Using the native worker, I get the error
RuntimeError: error: dirty exit (pid 2771 SIGSEGV (signal 11))
It happens every time I run recommendify when the item_id has been added in a set. Otherwise, it exits correctly with "item count is zero or one"
Any ideas what could cause this? What information do you need from me?
Turned native: true, and Recommendify would fail silently w/ Jaccard similarity instead of throwing some error when native extensions are failing. (if not built, I do get a proper error message: recommendify_native not found - you need to run rake build_native first)
Note that matrices are built but the final similarities hash in Redis is empty:
redis 127.0.0.1:6379[5]> hgetall recommendify:users:ccmatrix
Some debug output added shows exit output as: item count is zero or one <-- this error should be thrown
Further debugging shows that unless you use DB 0 in Redis from Ruby, the native code will fail since it uses the default DB. Will open a new issue or fork for this.
root@bt:~# msfupdate
[]
[] Attempting to update the Metasploit Framework...
[*]
fatal: 'upstream' does not appear to be a git repository
fatal: The remote end hung up unexpectedly
[] Attempting to add remote 'upstream' to your local git repository.
[] Added remote 'upstream' to your local git repository.
HEAD is now at 99b4620 Do final cleanup for sap_configservlet_exec_noauth
Already on 'master'
remote: Counting objects: 197, done.
remote: Compressing objects: 100% (88/88), done.
remote: Total 161 (delta 124), reused 108 (delta 71)
Receiving objects: 100% (161/161), 24.55 KiB, done.
Resolving deltas: 100% (124/124), completed with 19 local objects.
From git://github.com/rapid7/metasploit-framework
[new branch] Datacut-master -> upstream/Datacut-master
[new branch] creds-counter-cache -> upstream/creds-counter-cache
[new branch] feature/railgun/error_msg -> upstream/feature/railgun/error_msg
[new branch] master -> upstream/master
[new branch] release -> upstream/release
[new branch] unstable -> upstream/unstable
Updating 99b4620..6821c36
Fast-forward
Gemfile | 2 +-
Gemfile.lock | 4 +-
.../gather/apple_safari_webarchive_uxss.rb | 100 +++++---
.../unix/webapp/php_wordpress_total_cache.rb | 249 ++++++++++++++++++++
spec/lib/rex/post/meterpreter/packet_spec.rb | 246 ++++++++++----------
spec/lib/rex/proto/http/client_request_spec.rb | 8 +-
6 files changed, 442 insertions(+), 167 deletions(-)
create mode 100644 modules/exploits/unix/webapp/php_wordpress_total_cache.rb
[*] Updating gems...
Fetching gem metadata from http://rubygems.org/.........
Fetching gem metadata from http://rubygems.org/..
Using rake (10.0.4)
Using i18n (0.6.1)
Using multi_json (1.0.4)
Using activesupport (3.2.13)
Using builder (3.0.4)
Using activemodel (3.2.13)
Using arel (3.0.2)
Using tzinfo (0.3.37)
Using activerecord (3.2.13)
Using database_cleaner (0.9.1)
Using diff-lcs (1.2.2)
Using factory_girl (4.2.0)
Using json (1.7.7)
Installing pg (0.15.0) with native extensions
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
/opt/metasploit/ruby/bin/ruby extconf.rb
checking for pg_config... yes
Using config values from /opt/metasploit/postgresql/bin/pg_config
checking for libpq-fe.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/opt/metasploit/ruby/bin/ruby
--with-pg
--without-pg
--with-pg-dir
--without-pg-dir
--with-pg-include
--without-pg-include=${pg-dir}/include
--with-pg-lib
--without-pg-lib=${pg-dir}/lib
--with-pg-config
--without-pg-config
--with-pg_config
--without-pg_config
/opt/metasploit/ruby/lib/1.9.1/mkmf.rb:381:in try_do': The compiler failed to generate an executable file. (RuntimeError) You have to install development tools first. from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:506:in
try_cpp'
from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:970:in block in find_header' from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:790:in
block in checking_for'
from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:284:in block (2 levels) in postpone' from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:254:in
open'
from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:284:in block in postpone' from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:254:in
open'
from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:280:in postpone' from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:789:in
checking_for'
from /opt/metasploit/ruby/lib/ruby/1.9.1/mkmf.rb:969:in find_header' from extconf.rb:43:in
Gem files will remain installed in /opt/metasploit/ruby/lib/ruby/gems/1.9.1/gems/pg-0.15.0 for inspection.
Results logged to /opt/metasploit/ruby/lib/ruby/gems/1.9.1/gems/pg-0.15.0/ext/gem_make.out
An error occured while installing pg (0.15.0), and Bundler cannot continue.
Make sure that gem install pg -v '0.15.0'
succeeds before bundling.
am using windows xp as host machine and black track 5 r3 as guest
Hi,
Thanks for your effort
I am getting error message when I running the below commands in system, Please throw me light on the same.
vijayram@ubuntu:~/projects/recommendify/recommendify/doc$ ruby example.rb
example.rb:3:in `require': no such file to load -- redis (LoadError)
from example.rb:3
Thanks and Regards
Vijay
DELETED
The native processor has the redis connection information is hardcoded to 127.0.0.2:6379. This should be configurable from ruby. See recommendify.c line 59.
https://github.com/paulasmuth/recommendify/blob/master/src/recommendify.c#L59
Just wondering if there could be any more instructions on how to get started? Would be very helpful for ruby newbies like me.
Once you build the Recommendify gem, where do you go from there? Do you have to write a new model, like in the example?
Trying to copy that code into a model file gives the error "uninitialized constant Recommendify" - does not recognize it as a gem/module. Tips? Thanks! (p.s. running recommendify 0.2.3 on windows)
I put gem 'recommendify'
in my Gemfile and ran bundle
and got the following:
Installing recommendify (0.3.6) with native extensions
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
/Users/rahil/.rvm/rubies/ruby-1.9.2-p0/bin/ruby extconf.rb
make
mkdir -p ../bin
gcc -Wall recommendify.c -lhiredis -o ../bin/recommendify
recommendify.c:4:29: error: hiredis/hiredis.h: No such file or directory
recommendify.c: In function ‘main’:
recommendify.c:20: error: ‘redisContext’ undeclared (first use in this function)
recommendify.c:20: error: (Each undeclared identifier is reported only once
recommendify.c:20: error: for each function it appears in.)
recommendify.c:20: error: ‘c’ undeclared (first use in this function)
recommendify.c:21: error: ‘redisReply’ undeclared (first use in this function)
recommendify.c:21: error: ‘all_items’ undeclared (first use in this function)
recommendify.c:22: error: ‘reply’ undeclared (first use in this function)
recommendify.c:83: warning: implicit declaration of function ‘redisConnectWithTimeout’
recommendify.c:92: warning: implicit declaration of function ‘redisCommand’
recommendify.c:100: warning: implicit declaration of function ‘freeReplyObject’
recommendify.c:111: error: ‘REDIS_REPLY_ARRAY’ undeclared (first use in this function)
recommendify.c:161: warning: implicit declaration of function ‘redisAppendCommand’
recommendify.c:162: warning: implicit declaration of function ‘redisGetReply’
make: *** [build] Error 1
Gem files will remain installed in /Users/rahil/.rvm/gems/ruby-1.9.2-p0@myproject/gems/recommendify-0.3.6 for inspection.
Results logged to /Users/rahil/.rvm/gems/ruby-1.9.2-p0@myproject/gems/recommendify-0.3.6/ext/gem_make.out
An error occured while installing recommendify (0.3.6), and Bundler cannot continue.
Make sure that `gem install recommendify -v '0.3.6'` succeeds before bundling.
Then when I ran gem install recommendify -v '0.3.6'
I got the following:
Building native extensions. This could take a while...
ERROR: Error installing recommendify:
ERROR: Failed to build gem native extension.
/Users/rahil/.rvm/rubies/ruby-1.9.2-p0/bin/ruby extconf.rb
make
mkdir -p ../bin
gcc -Wall recommendify.c -lhiredis -o ../bin/recommendify
recommendify.c:4:29: error: hiredis/hiredis.h: No such file or directory
recommendify.c: In function ‘main’:
recommendify.c:20: error: ‘redisContext’ undeclared (first use in this function)
recommendify.c:20: error: (Each undeclared identifier is reported only once
recommendify.c:20: error: for each function it appears in.)
recommendify.c:20: error: ‘c’ undeclared (first use in this function)
recommendify.c:21: error: ‘redisReply’ undeclared (first use in this function)
recommendify.c:21: error: ‘all_items’ undeclared (first use in this function)
recommendify.c:22: error: ‘reply’ undeclared (first use in this function)
recommendify.c:83: warning: implicit declaration of function ‘redisConnectWithTimeout’
recommendify.c:92: warning: implicit declaration of function ‘redisCommand’
recommendify.c:100: warning: implicit declaration of function ‘freeReplyObject’
recommendify.c:111: error: ‘REDIS_REPLY_ARRAY’ undeclared (first use in this function)
recommendify.c:161: warning: implicit declaration of function ‘redisAppendCommand’
recommendify.c:162: warning: implicit declaration of function ‘redisGetReply’
make: *** [build] Error 1
Gem files will remain installed in /Users/rahil/.rvm/gems/ruby-1.9.2-p0@myproject/gems/recommendify-0.3.6 for inspection.
Results logged to /Users/rahil/.rvm/gems/ruby-1.9.2-p0@myproject/gems/recommendify-0.3.6/ext/gem_make.out
There's too many people running their own forks
In the method #serialize_item in similarity_matrix.rb, I found there could be a bug when converting similarity metrics into a string. the original code is like this
def serialize_item(item_id, max_precision=5)
items = @write_queue[item_id].to_a
items.sort!{ |a,b| b[1] <=> a[1] }
items = items[0..max_neighbors-1]
items = items.map{ |i,s| s>0 ? "#{i}:#{s.to_s[0..max_precision]}" : nil }
items.compact * "|"
end
the issue is at statement "s.to_s[0..max_precision]". For some cases, the value of s could be a fairly small floating point number. when converting it to string, Ruby will use scientific notation as the result of s.to_s. For example,
"8.442380751371887e-05"
After executing "s.to_s[0..max_precision]", given max_precision=5, the return string will become "8.4423", which causes the similarity result being enlarged incorrectly.
I would suggest to change "s.to_s[0..max_precision]" to
"sprint('%.#{max_precision}f", s)"
with sprintf, the above example will output "0.00008".
it is an edge case, there could be other better solution to it. but just want to share my findings on this issue.
Thank you very much for the great gem!
MetaSync
Hi,
I just tried to install recommendify on my machine but gem install always fails with the errors below during the build of the native extension. I tried to install on Mac OS 10.7, Archlinux and Debian and had no luck. So it seems to be an error on the end of the gem. ;-)
mkdir -p ../bin
gcc -Wall recommendify.c -lhiredis -o ../bin/recommendify
recommendify.c:4:29: error: hiredis/hiredis.h: No such file or directory
recommendify.c: In function ‘main’:
recommendify.c:20: error: ‘redisContext’ undeclared (first use in this function)
recommendify.c:20: error: (Each undeclared identifier is reported only once
recommendify.c:20: error: for each function it appears in.)
recommendify.c:20: error: ‘c’ undeclared (first use in this function)
recommendify.c:21: error: ‘redisReply’ undeclared (first use in this function)
recommendify.c:21: error: ‘all_items’ undeclared (first use in this function)
recommendify.c:22: error: ‘reply’ undeclared (first use in this function)
recommendify.c:83: warning: implicit declaration of function ‘redisConnectWithTimeout’
recommendify.c:92: warning: implicit declaration of function ‘redisCommand’
recommendify.c:100: warning: implicit declaration of function ‘freeReplyObject’
recommendify.c:111: error: ‘REDIS_REPLY_ARRAY’ undeclared (first use in this function)
recommendify.c:161: warning: implicit declaration of function ‘redisAppendCommand’
recommendify.c:162: warning: implicit declaration of function ‘redisGetReply’
make: *** [build] Error 1```
I'm getting this error. Can you help?
Thanks
Installing recommendify (0.3.6) with native extensions
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
/Users/xxx/.rvm/rubies/ruby-1.9.3-p125/bin/ruby extconf.rb
make
mkdir -p ../bin
gcc -Wall recommendify.c -lhiredis -o ../bin/recommendify
recommendify.c:4:29: error: hiredis/hiredis.h: No such file or directory
recommendify.c: In function ‘main’:
recommendify.c:20: error: ‘redisContext’ undeclared (first use in this function)
recommendify.c:20: error: (Each undeclared identifier is reported only once
recommendify.c:20: error: for each function it appears in.)
recommendify.c:20: error: ‘c’ undeclared (first use in this function)
recommendify.c:21: error: ‘redisReply’ undeclared (first use in this function)
recommendify.c:21: error: ‘all_items’ undeclared (first use in this function)
recommendify.c:22: error: ‘reply’ undeclared (first use in this function)
recommendify.c:83: warning: implicit declaration of function ‘redisConnectWithTimeout’
recommendify.c:92: warning: implicit declaration of function ‘redisCommand’
recommendify.c:100: warning: implicit declaration of function ‘freeReplyObject’
recommendify.c:111: error: ‘REDIS_REPLY_ARRAY’ undeclared (first use in this function)
recommendify.c:161: warning: implicit declaration of function ‘redisAppendCommand’
recommendify.c:162: warning: implicit declaration of function ‘redisGetReply’
make: *** [build] Error 1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.