Comments (6)
Actually I don't full understand your question. What is x, y, z? You mean that coordinate struct that has 3 float in it?. And second my benchmark test-cases are not the same test-cases with this repository so a comparison between them ( at least only with numbers ) is wrong. Because they are not tested with same data. If you want a fully compare two of them you have to create a benchmark with same data. Don't worry if systems are similar then I benchmark them and let you now :)
Note: this benchmark results in seconds, my results in nano second.
from jin.
This is my test case without calculation comparing to Jsoniter version in that benchmark.
I see, the parser read and consume 586MB into memory.
package main
import (
"fmt"
"io/ioutil"
"net"
"os"
"runtime"
"github.com/ecoshub/jin"
)
func notify(msg string) {
conn, err := net.Dial("tcp", "localhost:9001")
if err == nil {
fmt.Fprintf(conn, msg)
conn.Close()
}
}
func main() {
bytes, err := ioutil.ReadFile("1.json")
if err != nil {
panic(fmt.Sprintf("%v", err))
}
notify(fmt.Sprintf("%s\t%d", runtime.Compiler, os.Getpid()))
prs, err := jin.Parse(bytes)
if err != nil {
fmt.Println(err)
return
}
_ = prs
// x, y, z := 0.0, 0.0, 0.0
// for _, coord := range jobj.Coordinates {
// x += coord.X
// y += coord.Y
// z += coord.Z
// }
// fmt.Printf("%.8f\n%.8f\n%.8f\n", x/len, y/len, z/len)
notify("stop")
}
from jin.
Okay. I going to analyze it thanks for your respond.
from jin.
No problem, this has also been merge a while ago, it consume less memory
kostya/benchmarks#234
For another reference:
kostya/benchmarks#232
from jin.
Yes I have finally finish to analyze and found some noticeable things.
First in this test file https://github.com/kostya/benchmarks/blob/master/json/test_jsoniter.go it is not a raw byte parsing it uses this 'Decode' function. It is pretty much similar to Marshal()
function in encoder/json
. But Jin has no support for 'Marshal/Decoder' functions that works with 'Struct' types (At least for now).
Second It really consumes 465.90 MB of memory for 106.3 MB test-case data, in my benchmark.
Thank you so much for bring this up I will work on it.
Here is my benchmark results for your test-case '1.json'
(1.json is 106.3 MB has 2 keyword, first keyword is an array that has 524,288 element.)
JsoniteratorParse 2 864534010 ns/op 190838500 B/op 4182647 allocs/op
JinParseParse 2 827878780 ns/op 488532972 B/op 9961426 allocs/op
In parsing with Jin is slightly faster but it consumes much more memory (468.90 MB) than jsoniter.
and I notice somethin. Things getting interesting in Parse() + Get() benchmark.
JsoniteratorGet 1 4697474480 ns/op 882330784 B/op 23004635 allocs/op
JinParseGet 2 824512052 ns/op 488532376 B/op 9961423 allocs/op
In Benchmark of Parse() + Get() function I parse the data and try to access 0. 262,144. and 524,287. element of "coordinates" keyword. (Basically first, middle and last values.)
Jin is still consuming 465.90 MB of memory but jsoninter now consumes 841.46 MB. that is huge. And respond time is 4.7 seconds. Jin response time is 0.82 seconds it is 5.7 times faster.
I don't know maybe just parsing doesn't mean anything.
here is the benchmark file.
package kostyaBench
import (
"github.com/ecoshub/penman"
"github.com/ecoshub/jin"
"github.com/json-iterator/go"
"testing"
)
var json []byte = penman.Read("/tmp/1.json")
func BenchmarkJsoniteratorParse(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
prs := jsoniter.Get(json)
nop(prs)
}
}
func BenchmarkJinParseParse(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
prs, _ := jin.Parse(json)
nop(prs)
}
}
func BenchmarkJsoniteratorGet(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
prs := jsoniter.Get(json)
prs.Get("coordinates", 0)
prs.Get("coordinates", 262144)
prs.Get("coordinates", 524287)
}
}
func BenchmarkJinParseGet(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
prs, _ := jin.Parse(json)
prs.Get("coordinates", "0")
prs.Get("coordinates", "262144")
prs.Get("coordinates", "524287")
}
}
func nop(_ ...interface{}) {}
from jin.
Thanks for the guidance and It's awesome to read how both fare, probably I have opposite result on macOS Catalina, I hope it's useful for your reference and will be keen to further test.
Jsoniter parser using unmarshal with Penman ran a full benchmark and completed in 1.15s.
jin.Parse(json) with Penman load 1.json was 1.30s.
from jin.
Related Issues (16)
- need json types as constant
- need a decent error comparisson function
- separation of benchmark files to different branch HOT 1
- new empty json object constructor (JO)
- GetType func missing json length check
- tests are failing with a 'node error' HOT 1
- AddKeyValueString is not adding quote HOT 1
- setting 'nil' element with 'store' is failing
- walk func does not consider callback return params
- [suggestion] need AddKeyValueInt64 and AddKeyValueInt32
- [suggestion] json validation
- null json return error
- Setting null string is returning error
- suggestion: Set functions not setting a value if key not exists.
- There is no "toJSON" function for parser
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jin.