Product Roadmap about dgraph HOT 70 CLOSED

It'd be cool if the DB could serve GraphiQL. This would be perfect for demos + internal spelunking through data.

from dgraph.

Support tinkerpop? https://tinkerpop.apache.org

from dgraph.

Cypher support will be great indeed for the application layer. I also see that Tinkerpop support is on the Roadmap after 1.0, but I would have prefered to see SPARQL instead. Since the data model for Dgraph is already Triple/Nquad based, it would be logic and convenient to expose a SPARQL 1.1 endpoint out of the box, in addition to the GraphQL endpoint. SPARQL support is very important to fully interact with the semantic web, without the need to take a full Dgraph (triples) backup which then again needs to be imported in a separate TripleStore such as BlazeGraph to be able to run SPARQL queries against the Dgraph data.

from dgraph.

Hey @joeblew99 , others,

It is very important for us to figure out a revenue model, if we want to continue working on Dgraph three years down the lane. That's the only way we can keep supporting the salaries of people who are dedicated their lives on Dgraph. Without money, Dgraph would stop spinning (think RethinkDB).

Keeping that in mind, we have divided the project into what we consider open core and what we consider commercial. Anything that a startup needs to run their services and scale is part of the open core. This includes all of the current functionality and running Dgraph distributedly (which is not for e.g., allowed by Neo4j).

Anything that's an advanced feature would then be put into the commercial, closed-source version of Dgraph. This means: ACLs, cluster management, multi-tenancy, auto-scaling on Amazon, Google, etc.

This is a delicate balance between giving startups the same (I know it's better but I want to stay humble) level of technology built within the walls of big companies, like Google, Facebook, etc.; while also finding a way to monetization to make the company profitable and keep hiring the top engineers we can find.

Note that for small companies, which are pre-revenue, we would most likely make the enterprise version free of cost (free, but not open). But, if a company is making revenue and benefitting from Dgraph, we do expect them to pay. That's what is going to allow us to keep our engineers working for Dgraph, in a very competitive market where giants can pay up a huge sum to entice engineers. So, their salaries have to be competitive and they must feel that this company is not going to vaporize, and is going to be around 10 years from now.

from dgraph.

@jcebXD -- Yeah, Cypher is something we'll fully support.

from dgraph.

While playing with DGraph, there is one thing that doesn't feel quite right to me :
DGraph returns JSON format (which is great!), GraphQL queries also uses JSON format (without values) which is also great! But GraphQL mutations only supports NQuad format?
I would like to see mutations also to be consistent in using JSON. As a matter of fact, there is already a W3C compliant RDF serialization format called JSON-LD: http://json-ld.org/

Accepting valid JSON-LD in mutations will also greatly improve developer experience using DGraph, since application logic these days all generate or serialize into JSON before persisting to a database. Manual JSON to NQuad conversion can then be eliminated from the application layer and moved to the database layer.

I would not suggest to remove support for NQuad mutations enterily (sometimes it can come in handy), But to add additional support for JSON-LD. I believe NQuad is still the way to go for massive batch imports, but JSON-LD is preferable for use in mutation queries, i guess.

from dgraph.

I suggest also support cypher for queries, to avoid the queries migration from neo4j users.

from dgraph.

One thing that's clear to me: Dgraph can only survive if we are 10x better than Neo4j and others. If we genuinely didn't believe that, we'd be instead working for some big company, 9-5 and getting a nice fat check at the end of the year. So, making Dgraph the best graph database out there is the top priority. And for that, we need to hire and retain the best engineers.

I think Firefox is a bad example. They were thriving when Google was paying (donating?) them a big sum of money; until Google decided to stop that and focus on Chrome. Since then, Firefox revenue has steadily declined, while Chrome has taken over the world. At this point, Firefox can't afford to get the top talent working for them -- while Google (/Chrome) continues to recruit the best and brightest, which in turn makes Chrome even better.
http://howdoesitmakemoney.com/how-does-mozilla-firefox-make-money/

Consultancy is great, and we intend to do that, but consultancy alone is not going to make a company significantly profitable (more consultancy requires more human power). All top companies today are profitable due to selling software (Google, Microsoft, Facebook, etc.).

P.S. I don't intend this to become a thread of its own; so feel free to email me if anyone has concerns: manish, dgraph.io.

from dgraph.

Aggregations are not on the roadmap for 1.0, and neither after 1.0 ?
The ability to perform aggregations (COUNT, SUM, AVG, MIN, MAX, GROUP BY, DISTINCT,...) is clearly one of the main reasons you store data in a database in the first place and all acient and modern databases and database languages (SQL, CYPHER, SPARQL,...) support it. Why are they missing from the roadmap for DGraph?

Also for a new product anno 2017 it also make sense to have built in support for subscriptions (changefeeds ala RethinkDB). since many of the applications we built today needs support for real-time and push features...

DGraph is a great and promising database project, but without these features one would still be forced to move to other open-source alternatives such as ArangoDB which at this moment is the only serious open-source "competitor" for DGraph.

from dgraph.

We do intend to have a real-time feed, where mutations can cause certain queries to be re-run based on some dependency chart; and the results streamed back. We just haven't yet come across the right use case to do it. But, that's definitely a feature I'd love to have in Dgraph.

So, if you have specific use case which this feature would help with, ping me.

from dgraph.

@jimanvlad : We'll start working on Cypher right after v1.0.

@Senmumu : In addition to what @pawanrawal said, just wanted to mention that we're investing heavily in the web UI. It would be broken out of the Dgraph binary, and be a standalone server, capable of doing a lot more things than what it does today. More on that in the next few months.

from dgraph.

About the JS client. It would be interesting to have "seemingly native" support from you. Because I believe that the JavaScript community would adopt this project if it speak their language, even though it is super easy to adopt and "Go Lang" is simple. A lot of people ignore by the simple fact of not having "native support" for the language they are used to. Did u get it?

I myself have already thought a thousand ways to reconcile this database with my project. Currently I use RedisGraph, but it takes lot of work to maintain along with Redis and GraphQL. "Typing" meaning.

In this case, the easiest would be to plan my Schema to do in my resolvers a fecthing plan and that's it. I knew this project exactly 30 minutes ago. That's what I thought of right away.

And I'm only here for the simple fact that this project promises me speed (See I use Redis), Graph and is GraphQL like. The best of both worlds. From the little that I have read and seen, I believe that this project is perfect for me.

Pls "Suport Js"

Cheers

from dgraph.

@Didericis @MichelDiz -- By JS client, I meant a Dgraph specific JS client which would talk to Dgraph over GRPC. We're adding support for transactions, and hence, all the mutable queries now need to happen via a client.

Apollo / GraphQL compatibility is something we intend to look into past v1.0, once our query language is stable, and we are clear that it fulfills most functionality required by our users. At that point, we can figure out a way to bridge the gap between GraphQL and our version of it.

@joeblew99 : There is an issue about something similar: #824

from dgraph.

@manishrjain Are the proprietry plugins going to be closed source.
Sorry to aks this, but ACL's for example not being part of the open source offering ? Its pretty harsh.

from dgraph.

Yes I can only agree. It would be cool if you guys support JavaScript.

from dgraph.

We're working towards the hosted service quickly. In fact, starting to onboard a few early clients -- so feel free to reach out, if someone wants to go into private beta.

Cypher and other languages support are still slated for this year -- but we're a bit bogged down by all the activity around Dgraph since v1.0 release; doubling the team size to surface back and continue shipping features.

from dgraph.

@gedw99 -- We're not currently thinking about mobile. All our efforts are focused on production for large scale deployments.

from dgraph.

You guys are lightning fast ;-) Great to hear that you're already working on this.

from dgraph.

@MichelDiz Apollo looks at queries in order to determine what's cached/how to deduplicate things/how to normalize the results, so it needs to understand GraphQL. I think GraphQL+- is similar enough that a lot of Apollo could probably interpret GraphQL+- the same way without many changes, but I'm not sure.

Another solution is to convert GraphQL+- to GraphQL to expose something that Apollo could work with (the "tricky" endpoint you mentioned), which is what this does: http://dpeek.com/dgraphql/. I haven't tried hooking it up yet, but it looks like it should work. I just got excited about talk of a javascript client for Dgraph. I worry about GraphQL+- and GraphQL drifting apart and/or Apollo getting fancier to the point where it isn't easy to hook them up. It'd be cool if there was some sort of frontend client to help with caching/efficient querying specifically with GraphQL+- compatibility in mind.

from dgraph.

from dgraph.

Thanks for all your comments! It gives me a lot of joy to see how far Dgraph has progressed since this first issue on Github. Think it's time to close this issue. We're still listening to what our users want, so go on and tell us what you'd like to see built in Dgraph in the new 2019 roadmap here: #2894 .

Superseded by #1966 and #2894 .

from dgraph.

I'm running a DGraph server holding Freebase Film data, which you can query like so: curl dgraph.xyz/query -XPOST -d '{}'. An example of the query is in the README.

from dgraph.

from dgraph.

Javascript client? If there's a need for it, we can create one. But, the current flow is pretty straightforward, you query via POST, and get the results. So, hard to determine how would this client help.

from dgraph.

@voxadam currently rocksdb was removed from master and now it is using badger as storage, I guess the next release will contain badger as default

from dgraph.

Yeah, JS support is on our radar. We do want to support it; we're looking for an engineer with good JS skills to help us do this -- so if someone wants to help us out, here's the link:

https://boards.greenhouse.io/dgraph#.WcTDa3UjG0o

from dgraph.

Just to clarify, "JS client" means something like apollo, correct? I would love some sort of modified apollo client that works with GraphQL+-. Is the query language stable enough for that sort of a project?

from dgraph.

@pmualaba : Dgraph v0.9 adds distributed transactions, and fully supports JSON as both input and output format. It makes interaction with Dgraph a lot easier.

from dgraph.

Hey @Senmumu

Yes, Alter and Mutate operations would be possible from the Dgraph Browser from the next release. It has already been implemented and merged to master.

from dgraph.

Two questions:

1. Multi-homing support: Does this relate to datacenter / rack awareness for geo-distributed setups?
2. Any update on a hosted Dgraph service? Q1 2018 still is the period I'm hearing about.

from dgraph.

Tinkerpop would happen with Gremlin support. We'll do that after v1.0 gets released. This roadmap is largely focused on v1.0.

from dgraph.

For gossip discovery, I've found hashicorp/memberlist to work quite well.

from dgraph.

@F21: Instead of using Gossip protocol, we used the already implemented RAFT protocol for keeping track of membership. In terms of discovery, our cluster requires knowing at least one node in the cluster; any healthy node would do.

from dgraph.

Is there any interest in supporting mobile ?
Running a dgrpah DB on mobiles using gomobile is possible. Even with the CGO aspect of rocksdb.
I currently run boltDB on the servers and the clients with golang. It is nice to have that reuse.

For example i build apps for mobile, desktop using golang with this: https://github.com/therecipe/qt
It uses CGO heavily, and works well.

from dgraph.

@pmualaba: We're working on aggregate functions already! #532 It was missing on the roadmap. Thanks for pointing it out. We've added it now.

from dgraph.

Thanks for the feedback, @pmualaba. JSON-LD seems like a nice format for mutations. We have @ashishnegi working on it.

from dgraph.

@manishrjain i agree with you.
Many startups that were making new database systems failed due to lack of revenue. We dont want another rethinkdb, and the open core model makes sense.

But, everyone needs ACLS. I would think about if ACLS are in the open core of not.
The other stuff is for commercial for sure - so i agree with that.

But ACLS is so fundamental to any system you run always.
I mean any open source project that wants to build a forum or a chat system needs ACLS.

from dgraph.

What are your thoughts on subscriptions? Do you plan on supporting libraries for languages other than Go (and Python and Java)?

from dgraph.

We have community run libraries for Python and Java. You can find their repos in dgraph-io org in Github.

Not sure what you mean about subscriptions.

from dgraph.

I mean GraphQL Subscriptions (graphql/graphql-spec#267).

from dgraph.

Thanks @manishrjain, that would indeed be a great addition to an already impressive feature set. I'll be sure to do so should the need arise. Thank you.

Do you have any plans for a JavaScript client?

from dgraph.

How to deploy the distributed server?
And after a 2 days running test , we found a serious memory leak. Is it caused by RAFT?

from dgraph.

@TwinXu You can find the documentation for running distributed server at.
https://docs.dgraph.io/v0.7.5/deploy/#multiple-instances

This issue is for tracking product roadmap, can you please file an issue with the details of memory leak.
There shouldn't be any memory leak due to RAFT. It would be great if you could provide some more details about the memory leak.(What all you did, how much memory leak did you notice, what was the configuration with which you started the server). You might notice that the memory consumed by the server increases if your query touches new data until you hit the stw ram limit. That is not memory leak, it would be evicted eventually when needed.

from dgraph.

@janardhan1993 thank you for your reply.
I will new a issue for the detail of memory leak~

from dgraph.

Re: usefulness of a javascript/typescript client: I think that providing a light wrapper around gRPC functionality (similar to the java client) would be helpful. I would imagine that a package like this will land on NPM anyway, so it is really just a question of whether it is maintained by Dgraph or some other person.

from dgraph.

Hey guys -- am working on a Dgraph client for .NET. Is it safe to assume that the API / gRPC shape is going to remain in flux until 1.0? Do we have any milestones on when to expect the api contract to stabilize?

from dgraph.

No mention of Badger? Are you still planning on migrating from RocksDB to Badger by 0.8?

from dgraph.

Do we have any milestones on when to expect the api contract to stabilize?

We're now starting to actively think about v1.0, and stabilizing the APIs.

from dgraph.

Means "any support" native - Even an Apollo like. But what I really said was to create a Javascript context-friendly support model - To create applications in JS using NodeJS for example. Do not changing the core in Go Lang, which would be impossible and meaningless.

I have already cataloged some projects that support Dgraph. However, they take a long time to update as the Dgraph team make changes and add features. It's tricky for them to keep something of the sort practically alone. I'm helping with my limitations.

But wait second, Apollo does not handle data from a GraphQL API just through "Json" results? I do not think it would be complicated to leave Dgraph compatible with Apollo. It would make an exclusive endpoint to "trick" Apollo. Making "him" think it's consuming GraphQL. It would not be possible?

from dgraph.

On another topic..

Would there be a possibility for change notification on queries ?

I use NATS for this currently btw.

Thanks for a awesome dB !!!

from dgraph.

@manishrjain

Anything that's an advanced feature would then be put into the commercial, closed-source version of Dgraph. This means: ACLs, cluster management, multi-tenancy, auto-scaling on Amazon, Google, etc.

I think ACL is very important not to be considered part of the open source core.

graphcool offers ACL out of the box along with authentication/authorization, subscriptions and other features.

They offer a hosted service Graphcool Cloud and that is how the monetize; maybe a hosted service would allow you to monetize without sacrificing essential features all devs and small startups need.

Just a thought.

from dgraph.

@sebastianmacias : Noted. Once we're past v1.0, we'll reconsider this.

from dgraph.

@manishrjain sounds good. Thanks

from dgraph.

Can you guys write a demo example for distributed transactions.
Can be simulated with dockers.
Really curious how the linearizability performs. Lots of gremlins

from dgraph.

They will produce some examples soon. Follow the news in both the blog and the forum.

from dgraph.

from dgraph.

will you guys consider the implementation of ALTER in web UI ?

from dgraph.

Has the timeline for Cypher support changed?

from dgraph.

Any updates on Cypher support and Hosted service?

from dgraph.

@manishrjain any update on timeline for hosted service? I'd be interested in private beta.

from dgraph.

Are there any plans for supporting streaming/continuous query type operations?

from dgraph.

At the list at the top you have backup, but no restore. Does the community version support restore ? I need to test all DR aspects before recommendation to our team

from dgraph.

How is the initial pricing thought to be for the hosted service in comparison to Amazon Neptune?

from dgraph.

We're not doing hosted service for now. Backup and restore would be one feature. There's no point of backup, without any way to restore :-). But, there's already a way to export the data in NQuad format and import it back.

from dgraph.

from dgraph.

@gedw99 The backup feature still not exist. So, for that there's no restore. But you have "Export" and ou can re-import this exported file into a new instance. See more details here: https://docs.dgraph.io/deploy#export-database

take a look on:
https://docs.dgraph.io/deploy#bulk-loader
https://docs.dgraph.io/deploy#live-loader

from dgraph.

All Dgraph code base is open. We currently don't have any enterprise features.

from dgraph.

I really like this project, but there're two missing features block me from using it at our development environment, one is user authentication, the other is Tinkerpop/gremlin support. If you could prioritize these two features, that would be very helpful!

from dgraph.

Gremlin is on the roadmap. We have ideas around user auth, but for now, you could use the password type to store user passwords and use that to authenticate them.

from dgraph.

I'm learning Dgraph. A smarter "Query Graphical User Interface" would be very helpful when exploring the graph database, like Grakn.ai has. And moreover, if there is a "How to do reasoning" section in Help Doc, that would be wonderful. One more question: does dgraph support hypergraphs? Thanks!

from dgraph.