Comments (4)
Thanks, can you confirm empirically that 1 is better than 0?
If so, I'll change the default along with other udpates this week.
from nodevectors.
It is for my graphs, but I'm not sure if this is always the case. There is some discussion about which one is better, but in the context of NLP. I couldn't find anyone discussing that in the context of graph embeddings.
Anyway, I suggest you to not only use 1 as the default, but also force w2vparams["sg"]
to 1 instead of leaving this decision to the programmer, or make this a separate parameter in the same way you did with w2vparams["workers"]
. The reason is that it is very easy for the programmer to forget to set this parameter when he wants to customize other w2v parameters (this was exactly how I stumbled upon this). And node2vec was built explicitly with skip-gram in mind.
from nodevectors.
Check the reference below for skip-gram vs CBOW. The quality difference between the two seems to be the fault of a longstanding implementation error in the original word2vec and Gensim implementations.
İrsoy, Ozan, Adrian Benton, and Karl Stratos. “Koan: A Corrected CBOW Implementation.” ArXiv:2012.15332 [Cs, Stat], December 30, 2020. http://arxiv.org/abs/2012.15332.
from nodevectors.
Just passing through, noticed this issue in the course of answering someone's Node2Vec/Word2Vec interaction question, thought I'd mention:
- In the decade since word2vec arrived on the scene, I've not seen a strong consensus emerge on whether SG or CBOW is generally better. There are some reasons to suspect CBOW's batching of more operations allows more data/training in the same amount of time... but for some datasets/purposes/parameters/evaluations, SG still does better. So while you might want to default to SG if the original specification of 'Node2Vec' implied it creating a user expectation for that mode, perhaps, but otherwise there may not be a strong reason to nudge users any particular way - leaving it up to the advanced optimizers to dig deeper.
- I'm not convinced that the 'Koan' paper authors have identified anything 'incorrect' about usual word2vec implementations (including the original word2vec code, & later Facebook FastText code, by some of the original word2vec paper authors). Their paper shows a mix of both better & worse evaluations after their change (depending on the evaluation – & some of the claimed improvements might be due to other changes in the performance or behavior of their code, besides their reinterp of CBOW backpropagation. In particular, tinkering with Gensim
Word2Vec
parameterscbow_mean
,alpha
, & (new since their paper)shrink_windows
might match or exceed any benefits they've observed (on similar data/evals), without fully adopting their CBOW interpretation. See my longer comments in a Gensim issue for more details.
from nodevectors.
Related Issues (20)
- Embedding a VERY LARGE graph, upcoming? HOT 2
- When saving large graph, creating a temporary folder will cause the system disk resources to be exhausted. HOT 1
- Issue with gensim 4.0.0+ HOT 3
- is it possible to split n2v to generate walks only? HOT 4
- Setting value of seed to make Node2vec embedding repeatable. HOT 1
- Print training progression (node2vec)? HOT 1
- Continue fitting process HOT 2
- Has node2vec implementation been updated to use skip-gram as default? HOT 3
- About painting HOT 1
- defining random state or seed option parameters HOT 3
- Why is generating walks so slow with non-default parameters? HOT 3
- word2vec parameters changed HOT 3
- Problem with underlying Word2vec HOT 1
- G.mat got an asymmetric sparse matrix
- ProNE option: "inconsistent shapes" error
- Node2Vec:About the return_weight and neighbor_weight
- ProNE multithread HOT 1
- NetworkX 3.0 remove adj_matrix in version HOT 1
- Old parameter shows up in Word2Vec call
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nodevectors.