jayjacobs / verisr Goto Github PK

I am working on an incident tracking project for which I manually created a list of records using the Veris schema. I can import into r using json2veris but when I run the summary command it produces output like this:

7 incidents in this object.
actor action asset attribute
3:1 7:6 1:7
4:6 8:1

Expected output would be more like this:
actor action asset attribute
Malware:1 External:6 Unknown:7
Hacking:6 Internal:1

Any ideas what is wrong?

some dirs and repositories are loaded with names, others are not.

When json2veris attempts to pull down the schema it gives the following error:

Error in function (type, msg, asError = TRUE) :
SSL certificate problem: unable to get local issuer certificate

I attempted to pull down the schema manually using a getURL call that includes a 'cainfo' parameter. I then used the same fromJSON call to get the schema. But when I passed it in using the 'schema' paramter in json2veris I received another error.

Error in readLines(file, warn = FALSE) : 'con' is not a connection

Unfortunately that is the extent of my R knowledge.

Value in the loaded objects has names like:
victim.revenue.iso_currency_code."amount" : 226900000000

weird.

Hi, not sure if this project is still active. When I try to run json2veris, it crashes and prints the message below:

free(): double free detected in tcache 2
Aborted (core dumped)

Jay.. got your fixed code... thanks. i'm not sure but I think there may be some additional errors.
When I try and getenum on assets, attribute I get results like (again.. using the VCDB data)

v = json2veris("/Users/sbade1/Data/VerisR/VCDB/data/json")
acv <- getenum(v,"attribute.confidentiality")
acv <- getenum(v,"attribute.confidentiality")
Error in colSums(veris[filter, thisn, with = F]) : 'x' must be numeric
acv <- getenum(v,"attribute.confidentiality")
Error in colSums(veris[filter, thisn, with = F]) : 'x' must be numeric
asset <- getenum(v,"asset")
Error in colSums(veris[filter, thisn, with = F]) : 'x' must be numeric

I'm not sure if this is the right place, but I am trying to take your latest verisr code and the latest VCDB data, and reproduce the results from chapter 7 of your book.
First, when the data is loaded, a very large number of Warning messages are kicked out.
Warning found trying to set 2, "actor.External" for "TRUE"
length of assignment: 1
in 2 /Users/sbade1/Data/VerisR/singleVCDB/037B7432-60E7-4B1D-8904-0A1D513DB382.json
<simpleWarning in set(veris, i = i, j = x, value = paste(nfield[[x]], collapse = ",")): Coerced 'character' RHS to 'logical' to match the column's type. Either change the target column to 'character' first (by creating a new 'character' vector length 2 (nrows of entire table) and assign that; i.e. 'replace' column), or coerce RHS to 'logical' (e.g. 1L, NA_[real|integer]_, as.*, etc) to make your intent clear and for speed. Or, set the column type correctly up front when you create the table and stick to it, please.>

Second, summary(vcdb) gives (i am using a subset of the data - 2 incidents - for illustrative purposes)
gives.

summary(vcdb)
2 incidents in this object.
< table of extent 0 x 4 >

And enums for any level of property returns

x <- getenum(vcdb,"action")
print(x)
data frame with 0 columns and 0 rows

I tried to run the RMD file and consistently I have been rewarded with this error:

with=FALSE together with := was deprecated in v1.9.4 released Oct 2014. Please wrap the LHS of := with parentheses; e.g., DT[,(myVar):=sum(b),by=a] to assign to column name(s) held in variable myVar. See ?':=' for other examples. As warned in 2014, this is now a warning.with=FALSE together with := was deprecated in v1.9.4 released Oct 2014. Please wrap the LHS of := with parentheses; e.g., DT[,(myVar):=sum(b),by=a] to assign to column name(s) held in variable myVar. See ?':=' for other examples. As warned in 2014, this is now a warning.with=FALSE together with := was deprecated in v1.9.4 released Oct 2014. Please wrap the LHS of := with parentheses; e.g., DT[,(myVar):=sum(b),by=a] to assign to column name(s) held in variable myVar. See ?':=' for other examples. As warned in 2014, this is now a warning.with=FALSE together with := was deprecated in v1.9.4 released Oct 2014. Please wrap the LHS of := with parentheses; e.g., DT[,(myVar):=sum(b),by=a] to assign to column name(s) held in variable myVar. See ?':=' for other examples. As warned in 2014, this is now a warning.

I believe the library uses some deprecated DT call on some function.

You can even see this in the README. When you run summary(vcdb) at the bottom you get confidentiality, availability, confidentiality, and integrity.

For specific fields like industries, categories of action, actors and assets, employee count, etc. the code should be aware and create an ordered factor on values returned from those.
May want to create support functions to retrieve levels for things:
actionCatLevels
assetCatLevels
(or something without camel case)

R version 3.2.0 (2015-04-16)
versir package version: verisr_2.0.6

Failed to load vcdb instance [ vcdb <- json2veris(jsondir)] into a matrix:
> vmat <- veris2matrix(vcdb) Error: could not find function "veris2matrix"

when querying industry2 or industry3, return text in enumeration.
add an alias for "orgsize" and replace employee_count with just large/small
relabel asset varieties to put asset variety category in parenthesis behind label.

I have some scripts written around 2014-12-13 that used the json2veris function of the time:

json2veris <- function(dir=".") {
# create listing of files
jfiles <- unlist(sapply(dir, list.files, pattern = "json$", full.names=T))
jread <- function(jfile) {
doc <- fromJSON(file=jfile, method='C')
}
# now read them all
veris <- lapply(jfiles, jread)
# set my class
class(veris) <- "verisr"
veris
}

I see that it was renamed to json2veris.old() before being removed. Could we perhaps have some form of legacy support for this old function? Perhaps if the user sets schema="LEGACY" or something similar. I am also happy to clone your repo and incorporate it myself, if needed.

add the "orgsize" short cut into the getenum, and fix the formatting.

I'd like to first state that i am just exploring the R programming language, and its intricacies. Right now I am trying to explore ch.07 of Data Driven Security and follow the code from the book. I'm completely stumped on an error and was wondering if anyone can please provide some kind insight on the matter.
library(ggplot2)
library(scales)
library(rjson)
library(devtools)
install_github("verisr", "jayjacobs", force = TRUE)
library(verisr)

jsondir <- ("C:/Users/Kxnice/ch07/data/vcdb/")
vcdb <- json2veris(jsondir)

Error in fromJSON(json_str = x) : unexpected escaped character '\d' at pos 2

the or= will create errors with multiple values and this just needs to be tested more.

Hi

i tried to rung some verisr scripts and got the below error

could not find function "json2veris"

any solution please?

want a function to take in an object from getenum() and sort it properly

Hey! Following along with your (excellent) book, chapter 7....

When I try to run the json2veris command (listing 7-6), I get the following error:

> jsondir <- 'data/vcdb/'
> vcdb <- json2veris(jsondir)
Error in fromJSON(json_str = x) : 
  unexpected escaped character '\d' at pos 2

Digging a little deeper into your source code, in the json2veris function, if I try to straight-up load the schema with the rjson package, this is where the issue is:

> x <- getURL("https://raw.githubusercontent.com/vz-risk/veris/master/verisc-merged.json")
> lschema <- rjson::fromJSON(json_str=x)
Error in rjson::fromJSON(json_str = x) : 
  unexpected escaped character '\d' at pos 2

I suspect at some point the verisc-merged schema changed and now can no longer by read.

My workaround was to find the original schema in the GH history https://raw.githubusercontent.com/vz-risk/veris/86e76f1e77e07480c54ac9cb11642711c98e9cf0/verisc-merged.json, found from https://github.com/vz-risk/veris/commits/master/verisc-merged.json, download that to local, and then run the json2veris function with a specification to that schema:

jsonfile <- 'data/orig-verisc-merged.json'
vcdb <- json2veris(dir = 'data/vcdb/', schema = jsonfile)

No doubt there is a more recent version I could have checked to see if it worked, but I didn't care to do the binary search to figure out what was the most recent one to work.

The above ran (with a number of warnings), and I was able to reproduce some answers from your book.

I guess, however, the upshot is that the json2veris function doesn't like the current schema.

it was originally written to expand the amount of something when requested (like asset varieties or asset categories). But this get's weird when looking at the frequency. (4000+ user devices over 100 incidents causes proportion to be > 1).

Remove the inclusion of counts from general "getenum" query.