The sandpiper's discuss from hewlettpackard

Pairwise factors?

Hi,

This is not exactly an issue. I am using the PairwiseBP algorithm for HTTP log files. But unable to understand the factor values in edges. And in sample data also its not very much clear. Can you please provide a little clarity on how are we getting edge factors?

Thanks

java.lang.NoSuchMethodError: scala.runtime.IntRef.create(I)Lscala/runtime/IntRef;

Hi,

I tried running the BP algorithm with the provided example. I am using spark 1.6.1. Also, build the code using steps provided but getting following error:

java.lang.NoSuchMethodError: scala.runtime.IntRef.create(I)Lscala/runtime/IntRef;
	at sparkle.graph.PairwiseBP$.apply(PairwiseBP.scala:37)
	at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:43)
	at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:48)
	at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:50)
	at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:52)
	at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:54)
	at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:56)
	at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:58)
	at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:60)
	at $iwC$$iwC$$iwC$$iwC.<init>(<console>:62)
	at $iwC$$iwC$$iwC.<init>(<console>:64)
	at $iwC$$iwC.<init>(<console>:66)
	at $iwC.<init>(<console>:68)
	at <init>(<console>:70)
	at .<init>(<console>:74)
	at .<clinit>(<console>)
	at .<init>(<console>:7)
	at .<clinit>(<console>)
	at $print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
	at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)
	at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
	at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
	at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
	at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
	at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
	at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
	at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
	at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
	at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
	at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
	at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
	at org.apache.spark.repl.Main$.main(Main.scala:31)
	at org.apache.spark.repl.Main.main(Main.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Following is my code:

import sparkle.graph._

val variableFile = "/home/user/sandpiper/data/vertex4.txt"
val factorFile = "/home/user/sandpiper/data/edge4.txt"

val graph = PairwiseBP.loadPairwiseGraph(sc, variableFile, factorFile)

val maxIterations = 50
val epsilon = 1e-3
val beliefs = PairwiseBP(graph, maxIterations, epsilon)

Please help me regarding the same.

Are there any limitations with the input FG?

Thanks for your effort on this.

I am wondering if there are any limitations on the input FG in terms of size or shape? I modified your input FG to have few factors, it fails with null pointers at the point of loading the graph.

Also, would be able to tell me how you decide the factor values in the input FG for BP?

Where are the message destination Factor or Variable excluded in message aggregations?

sorry this is not an issue at all, i just didn't see any other way to communicate.

i am looking at the source code and it looks really great! it is a pleasure to read. but i am struggling to understand one thing: when a variable sends a message to a factor its some sort of aggregation over all messages the variable received from factors excluding the factor the message is about to be send to. similarly when a factor sends a message to a variable it is also an aggregation over messages received from variables excluding the variable the message is about to be send to. where in the code do i see this exclusion behavior? i have a hard time finding it.

thanks for you help! best, koert

hewlettpackard / sandpiper Goto Github PK

sandpiper's Issues

Pairwise factors?

java.lang.NoSuchMethodError: scala.runtime.IntRef.create(I)Lscala/runtime/IntRef;

Are there any limitations with the input FG?

Where are the message destination Factor or Variable excluded in message aggregations?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent