Comments (3)
@explanare has this locally setup I think.
from pyvene.
What functionality should this PR have for best integration with existing pyvene tools?
I have an implementation of SAE training working with transformerlens models (largely inspired by Neel Nandaโs code but adjusted such that experiments are easier to modify). Currently the Buffer class collects the activations from a layer with the model.run_with_cache
function, I presume there is an equivalent pyvene function that should be used instead? Other than that I think all the other code should currently be interoperable with pyvene.
from pyvene.
@smejak currently, pyvene supports activation collection as in the first two examples in https://github.com/stanfordnlp/pyvene/blob/main/pyvene_101.ipynb. Will this help?
from pyvene.
Related Issues (20)
- [P1] Adding tests for functions in `modeling_utils.py`
- [P1] Adding tests for interventions and their util functions
- [P2] Mac chip MPS mode support HOT 1
- [P1] Tutorial of Inference-time Intervention HOT 1
- Bug in BoundlessRotatedSpaceIntervention HOT 1
- [P1] Speed up training of multiple DAS interventions with caching
- [P1] Optionally remove the dependency of the config file HOT 1
- [P0] Upgrade tutorials to new API HOT 1
- [P1] Dynamic Intervention Scheduler
- [P2] Add a new huggingface collator for working with Pyvene models
- [P0] Make interface compatible with HF trainer
- [Bug]: pyvene.ai is taking too long to respond HOT 2
- [P0] Adding back GPT2 and other model supports
- [Bug]: Is this repo support MPT architecture ? i got error HOT 1
- [P0] `IntervenableModel.save()` doesn't save trained model parameters
- [P1] Tuned lens support
- [Bug]: Whether to support intervene_on_prompt=False HOT 2
- [Feature Request / Suggestion]: Capturing of the residual stream HOT 5
- [P0] Making `RepresentationConfig` a PretrainedConfig object
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyvene.