Comments (6)
Yes, this sound good.
To clarify, MLJModelInterface.jl remains the home for MLJ's basement-level model interface. I just mention LearnAPI.jl because that is where I hope to move to later.
So we should add two new abstract types to MLJModelInterface: Set <: Supervised
and ProbabilisticSet <: Supervised
. We already have Interval <: Supervised
. Would you be willing to make the PR? The additions go here and here.
from conformalprediction.jl.
Addressed in #21
@ablaom is this what you had in mind? Instead of ConformalSet <: MMI.Supervised
we'd ideally have ConformalSet <: MMI.Set
in the future.
from conformalprediction.jl.
Yeah, mostly:
- At present
Interval
is a subtype ofSupervised
. - I don't see a need for a separate
ConformalInterval <: Interval
subtype in MLJModelInterface. If it were useful to you, that could live in ConformalPrediction.jl, no? - I would make make
Set
andProbababilisticSet
subtypes ofSupervised
.
In other words, we currently have
Probabilistic <: Supervised
Deterministic <: Supervised
Interval <: Supervised
and I would propose extending this with
Set <: Supervised # if you think you even need this
ProbabilisticSet <: Supervised
(In the new LearnAPI.jl, we would add two target proxy types, Set
and ProbabilisticSet
. These are possible values of a trait target_proxy
and there would be no need to restrict to supervised models, but that's another story...)
from conformalprediction.jl.
Would that work for you?
from conformalprediction.jl.
Hi Anthony! Yes, absolutely that works. In fact, that's what I had in mind and also reflect the way things are currently implemented (I think the diagram needs some clarification perhaps).
Also, good point ProbabilisticSet <: Supervised
, have added a placeholder for this now in #26. To be clear, at this point we have the following:
"An abstract base type for conformal models that produce interval-valued predictions. This includes most conformal regression models."
abstract type ConformalInterval <: MMI.Interval end
"An abstract base type for conformal models that produce set-valued deterministic predictions. This includes most conformal classification models."
abstract type ConformalSet <: MMI.Supervised end # ideally we'd have MMI.Set
"An abstract base type for conformal models that produce set-valued probabilistic predictions. This includes most conformal classification models."
abstract type ConformalProbabilisticSet <: MMI.Supervised end # ideally we'd have MMI.ProbabilisticSet
"An abstract base type for conformal models that produce probabilistic predictions. This includes some conformal classifier like Venn-ABERS."
abstract type ConformalProbabilistic <: MMI.Probabilistic end
For set-valued predictions, I am at this point still just subtyping <: MMI.Supervised
, since the specific types still need to be added to LearnAPI.jl. In particular, for conformal models of type ConformalSet
currently produce prediction of type UnivariateFinite
(linking our related discussion on discourse).
I will close this issue now and open a new one related to set-valued predictions specifically.
Thanks!
from conformalprediction.jl.
Thanks @ablaom 👍🏽
Yes, I'll make the PR as soon as I can (have a few commitments these next two weeks, but will get to it eventually 😅)
from conformalprediction.jl.
Related Issues (20)
- Conformal Training examples HOT 2
- Support for thresholding predictive distributions as explained in Section 2.4 of the tutorial HOT 2
- Conformal Bayes through 'add-one-in' importance sampling
- .vscode folder HOT 1
- Add Aqua.jl
- Add parallelizer field to all models
- Adaptive Inductive Classification broken? HOT 2
- Move to adjusted quantile HOT 1
- Class-Conditional CP with many classes
- Treat data as artifacts
- JuliaCon pres
- Add format check to CI
- Add support for RAPS
- [Refactor] Separate module for TS
- Revisit sample correction
- Move plot methods to TaijaPlotting.jl
- Add TaijaPlotting to docs env HOT 1
- Add support for 1.6 HOT 1
- readme Quick Tour notebook: "Could not fetch rendered notebook or notebook source." HOT 5
- Conformal Training
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from conformalprediction.jl.