Comments (7)
#38 should do what has been discussed.
from mlflowclient.jl.
To improve the project testability, I suggest to use BrokenRecord.jl. It brings us a "cassette" approach to record the HTTP requests, removing the necessity of having a mlflow
instance running on our local machine.
However, this can include a new environment variable USE_TEST_RECORDS
, because testing using a true server is still being required for the CI job (and locally if required).
from mlflowclient.jl.
@ablaom do you think is a good idea to replicate the environment variable approach implemented in MLJFlow testing suite? I think it can ensure the user to run all the tests correctly.
from mlflowclient.jl.
Sure, unless you have a better suggestion.
from mlflowclient.jl.
When the initial development happened, the assumption was that the developer would have a local mlflow instance running prior to executing the tests. Moreover, it was also up to restart (wipe out) the mlflow instance prior to running the tests another time. This helped with debugging (i.e. final state remains in mlflow if tests fail and developer can debug) and it was also meaningful because actual functionality was tested against an actual instance. The same setup we have in the github CI job - the mlflow instance is actually a separate container that is spun up alongside the test container.
Another way to go would be to mock the mlflow endpoints. This will not require an mlflow instance at all. That's probably the correct way in general and it will allow Pkg.test()
to execute without any service dependencies.
from mlflowclient.jl.
Helpful comments, @deyandyankov, thank you.
Another way to go would be to mock the mlflow endpoints. This will not require an mlflow instance at all. That's probably the correct way in general and it will allow Pkg.test() to execute without any service dependencies.
Do you mean a fully functioning mock up of an mlflow service? This would have these drawbacks:
- A bit of work to set up, no?
- Possibility that bugs in the mock mean some wrong behaviour of MLFlowClient.jl is not caught.
- Adds a maintenance burden to testing for breaking releases of mlflow
As maintenance on this site has become exceedingly thin, this doesn't make this realistic option just now, in my view. (Side note: I did actually created a baby mock-up for the POC at #41.)
@deyandyankov I might have missed something, but is there anything proposed in #38 that upsets the testing workflow you just described?
from mlflowclient.jl.
This sounds like a nice idea!
from mlflowclient.jl.
Related Issues (20)
- implement listartifacts
- createrun fails when HTTP response dictionary contains Int64 values HOT 8
- Incorrect start time displayed on MLFlow dashboard HOT 1
- Extending support for remote MLFlow server
- How to log model? HOT 4
- Improve tag definition
- Attributes such as model and dataset
- Support asynchronous logging HOT 16
- Authenticating via REST API HOT 4
- Integrate with DagsHub's MLflow API HOT 6
- Issue to trigger new releases HOT 6
- Proposal to buffer service requests HOT 2
- main doesn't pass it's own test HOT 3
- Support GitLabs MLFlow backend HOT 1
- Artificact upload for remote instances HOT 2
- Can't create experiment with v0.5.1 HOT 1
- `artifact_location` default is not being set
- Document how to set tags when creating an experiment HOT 3
- Update to the 2.x API HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mlflowclient.jl.