Comments (16)
- graph-*: looks ok
- resource-manager should be:
links:
- cassandra
- reverse-proxy
depends_on:
- graph-typesystem
- keycloak
- storage should be:
links:
- cassandra
- reverse-proxy
depends_on:
- graph-typesystem
- graph-mutation
- resource-manager
- keycloak
- explorer should be:
links:
- cassandra
- reverse-proxy
depends_on:
- graph-typesystem
- keycloak
from renku.
Should depend-on
specify services that are needed for successful execution of function or simply what is needed to start the service? I think the latter... and in this case:
- resource-manager and storage don't depend on keycloak -- they will start and work without it (or they should, anyway)
- does storage need to see the other services to start?
from renku.
resource-manager won't start without keycloak, storage will, but will 500 at every request.
Also, what's the point of starting without ensuring functional behavior?
from renku.
Well, take the deployer as an example. If it can't add vertices to the knowledge graph, should it refuse to create and execute contexts? I would say no -- it should just acknowledge that the KG is unavailable and that the state of the executions will be synced later. It doesn't actually do this now, but it probably should eventually. I'm not sure this pattern makes sense for other services that's why I asked the question.
from renku.
the storage uses keycloak to get the public key to validate the incoming tokens. (as discussed with leafty, this could be made optional as the user credentials are anyway checked by the RM)
the storage uses the graph mutation to populate the graph when reading/writing files/buckets. This is currently mandatory as it checks if it was correctly added before returning the permission token.
the storage uses the RM, obviously for managing the permissions.
the storage doesn't use the type-system as far as I know? or is it a transitive dependency?
from renku.
@ebiiii typesystem is needed when you use vertexReader
or EdgeReader
.
As it needs datatypes and such for reconstruction objects from janusgraph.
from renku.
does the RM need KC because it fetches the keys? If the idea is that the RM should work with other identity providers, should it not just be given the public keys to check the signatures?
For other services, they should just rely on the RM to check the tokens IMHO
from renku.
it should just acknowledge that the KG is unavailable and that the state of the executions will be synced later.
This is contrary to the current vision of the platform. Please synchronise with @erbou on this topic. The deployer behaving like this will break lineage and linkage done by other services when accessed from within the deployed application.
from renku.
I understand that -- we've had these discussions before. I'm just trying to clarify what actually depends on what and why.
from renku.
In my view, depends_on
specifies what the service needs to start. In many cases, the hard dependency is the db because the internally-used tables have to be initialized. Apart from that, the rest are soft dependencies. Yes, there will be runtime errors but the service won't crash. The runtime errors should be handled gracefully giving an informative message and let the service continue to run.
If we make soft dependencies into hard dependencies, I think it will make it really hard later on to decentralize/distribute and scale.
from renku.
As a user I want to be able to do
docker-compose up -d storage
and use it when it's ready. I DO NOT want to see any error message.
The docker-compose does absolutely not describe a production deployment, for which it is good/necessary to try failure scenarios where services have to deny access due to missing dependents.
from renku.
As a user I want to be able to do
docker-compose up -d storage
If you want to run the service separately then we should encourage people to use Dockerfile
and docker-compose.yml
from each repo respectively.
from renku.
ok I think we're splitting hairs here. Practically, there is no difference between typing docker-compose up -d
and docker-compose up -d storage
. However, by eliminating redundant links in the docker-compose.yml
we can force ourselves to write the services in a way that minimizes their interdependence. Therefore my insistence on figuring out the answer to e.g. "does storage really need KC?" Of course the docker-compose
doesn't yield a production environment, but it's our primary means of R&D so if we do it poorly here there is no reason to believe we'll do it better in production.
from renku.
Practically, there is no difference between typing docker-compose up -d and docker-compose up -d storage.
There is one if you don't bring up dependencies for docker-compose up storage
, as it won't work in that case.
from renku.
that's what I meant -- why would you want to bring up specifically just storage? You would just type docker-compose up
and be done with it.
from renku.
Dead conversation
from renku.
Related Issues (20)
- Running Renku image locally in Windows
- Feature `renv.lock` files in interactive web view HOT 2
- Navigation between Gitlab & Projects in Renkulab HOT 5
- Missing information for datasets imported from external repositories HOT 3
- switch to java based plantuml in docs
- Update "Get started" tutorial to match the UI style
- Cypress tests for Private project HOT 1
- Cypress tests to cover missing features
- Dicsonnected identities between Renku and Gitlab
- [ShapeUp] Let users pin projects to the dashboard
- [ShapeUp] Initial support spaces which group together other Renku entities
- [ShapeUp] Redesign Landing Page
- View all shared projects HOT 1
- [ShapeUp] Make Cloud Storage Read/Write + "Universal" Storage
- add csi-rclone to helm chart
- post-init issue HOT 1
- [ShapeUp] Renku 2.0 Sessions
- [ShapeUp] Renku User and Group Namespaces
- docs: add instructions solr/search
- Start session link automatically fetching LFS data
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from renku.