Is your feature request related to a problem? Please describe.
Currently, there seems to be no documentation on the dependencies required to run the Community Notes program, making it more difficult for normal users to match Twitter's internal environment and causes build errors along the way.

For example, the constants.py and process_data.py files now uses some numpy objects that are now considered deprecated and throws out errors due to that. Additionally, process_data.py assumes that Python will open TSV files using UTF-8, which is true for *nix environments, but not Windows.

Describe the solution you'd like
Document the necessary dependencies (along with the versions) to run Community Notes.

Describe alternatives you've considered
I've not considered any alternatives.

Additional context
No additional context available.

There is an account with a large number of followers that asked people to vote "useless" on the community notes attached to their posts.

https://twitter.com/tezheya/status/1724378192977498289

In fact, notes that were supposed to be "useful'' at one point were rendered "useless'' due to organized voting actions.

Can Community Notes withstand these vote-buying practices?

• Is it possible to write and rate community notes whose author is blocked by the tweet's author ?
• If not, tweet's authors and note's evaluators are always in the same clusters of ideas like political ideologies and religions and personal preferences.

I'm frustrated every few days when I open a useless notification from the communitynotes system.

X should un-send notifications from the communitynotes system once no action is needed:

1. The user has already voted on the note. (Admittedly, re-opening it is user error; an alternative solution for this sub-issue would be to make it easier to differentiate read from unread notifications.)
2. The relevant tweet has been deleted or hidden.

Screenshot of 2:

I have received 7 "Your Rating Impact has increased*" messages (all > 1 hr ago), but my rating is a 3.

Shown currently:

3 Rating Impact
Last update: 1 hour ago

---

6 < sic - ??? - What is the 6 for? > Want to start writing notes? Raise your Rating Impact to 5 or higher.

---

Ratings that helped a note earn the status of Helpful:
+3. <=FALSE

---

Ratings that helped a note reach the status of Not Helpful:

+0

---

Contrasting ratings:

-0

I assume I'm far from alone...?
Have I misread something? This seems to be a clear bug.
I've DM'd @communitynotes (starting 2/27), but DMs are ignored - not one response yet.

UPDATE: Aha!? Does https://communitynotes.twitter.com/guide/en/under-the-hood/contributor-scores#valid-ratings explain it? Then it's an in-app mis-messaging/documentation problem.

*As in You helped this note earn the status of Helpful, and it's now been seen by
over 250K people on Twitter. Your Rating Impact has increased. < content of Community Note >

In order to improve the speed at which important community notes get added and to help community noters write better notes, I'm curious if people have put some effort into using a mic of AI (language models) and more simple methods. I'd like to help with this if we can make it work economically.

For example, you could have a scaffolding approach that looks for specific words, which then feeds into an embedding for semantic similarity to contentious issues, and then finally into an LLM that ranks how important the tweet is to have a community and some additional context (through a web search and internal knowledge within the LLM) to help the community noter. I think there's a way to make this economically viable for companies.

Yes, companies, I want Community Notes to expand beyond X. Let's figure out how to connect it to YouTube. Why haven't other social media websites picked it up yet? If they care about truth, this would be a considerable step forward beyond. Notes like “this video is funded by x nation” or “this video talks about health info; go here to learn more” messages are simply not good enough. We need to improve the state of truth-seeking on the internet.

Not just that, as an AI Safety researcher, this is particularly important to me. Don't forget that we train language models on the internet! The more truthful your dataset is, the more truthful the models will be! Let's revamp the internet for truthfulness, and we'll subsequently improve truthfulness in our AI systems!!

Describe the bug
Clicking on a link leads to a not found page :(

To Reproduce
Steps to reproduce the behavior:

1. Go to https://communitynotes.twitter.com/guide/de/about/introduction
2. Go to section
   
3. Click link
4. Land on not found page
   https://communitynotes.twitter.com/guide/en/about/under-the-hood/download-data
   

Expected behavior
A page should be found.

additional stuff
on the english version of the page the link redirects to a valid page :)
https://communitynotes.twitter.com/guide/en/about/introduction#/about/introduction

Environment
Chrome Version 120.0.6099.71, German Language Culture

Describe the bug

Some time ago I got a notification I can write community notes.
Today I tried to write a simple note (one paragraph note + one paragraph "Source: LINK") and was told I may not be allowed to perform this action.

To Reproduce

1. Click on "Write a Community Note
2. Write a single-paragraph note followed by another paragraph ("Source: https://wikipedia.org/bla/blah")
3. Check "Yes" ("Did you link to sources...")
4. Click "Submit"

Expected behavior

Note gets submitted. Or, if I am not allowed to write community notes, don't let me see that "Write a Community Note" in post drop down menu.

Screenshots

Environment

• X for iPadOS v10.21.2
• iPadOS 16.7.4

Describe the bug
Cannot write notes while Impact Score is >5. See screenshots of profile scores

To Reproduce
Steps to reproduce the behavior:

1. Be a new CN user with Impact Score == 7
2. Attempt to write a note

Environment
Chrome 121.0.6167.86 and Firefox 122.0

Is your feature request related to a problem? Please describe.
The problem is that although media content now can get community notes but that is probably works well only on pictures. Video can contain many statements which needs separated notes on those.

Describe the solution you'd like
https://twitter.com/kcoleman/status/1671146865360814080?s=20
Could plausibly have notes attached to a transcript & associated with specific timestamps. Would not appear in the first livestream (takes non-zero time to get rating consensus) but most views happen async/later.

Describe alternatives you've considered

Additional context

Please if ya'll are online right now... go to twitter spaces

Describe the bug
Similar to this bug, the current scoring code is incompatible with the current data file format.

To Reproduce
Steps to reproduce the behavior:

1. Download source code
2. Download Birdwatch data files
3. Run python main.py
4. The error occurs while processing note scores

Expected behavior
Program should output scored notes

Screenshots

$ python main.py --epoch-millis 1686070222583 --headers -o ./
Timestamp of latest rating in data:  2023-05-28 00:31:49.395000
Timestamp of latest note in data:  2023-05-28 00:30:46.699000
total notes added to noteStatusHistory: 0
Preprocess Data: Filter misleading notes, starting with 4243086 ratings on 98052 notes
  Keeping 3077820 ratings on 67233 misleading notes
  Keeping 219979 ratings on 8448 deleted notes that were previously scored (in note status history)
  Removing 69184 ratings on 3421 older notes that aren't deleted, but are not-misleading.
  Removing 11994 ratings on 1727 notes that were deleted and not in note status history (e.g. old).
Num Ratings: 4161908, Num Unique Notes Rated: 92904, Num Unique Raters: 111825
Identifying core notes and ratings
  Total ratings: 4161908
  Ratings from user without modelingPopulation: 0
  Total notes: 112020
  Total notes with ratings: 92904
  Total core notes: 106666
  Total expansion notes: 5354
  Core ratings: 3803097
Filter notes and ratings with too few ratings
  After Filtering Notes w/less than 5 Ratings, Num Ratings: 3761285, Num Unique Notes Rated: 70266, Num Unique Raters: 78386
  After Filtering Raters w/less than 10 Notes, Num Ratings: 3621122, Num Unique Notes Rated: 70266, Num Unique Raters: 39076
  After Final Filtering of Notes w/less than 5 Ratings, Num Ratings: 3619999, Num Unique Notes Rated: 69975, Num Unique Raters: 39076
------------------
Users: 39076, Notes: 69975
cpu
epoch 0 6.531041145324707
TRAIN FIT LOSS:  6.066281795501709
epoch 50 0.12861372530460358
TRAIN FIT LOSS:  0.09712684154510498
epoch 100 0.11354893445968628
TRAIN FIT LOSS:  0.08736852556467056
Num epochs: 144
epoch 144 0.11346116662025452
TRAIN FIT LOSS:  0.08727223426103592
Global Intercept:  0.15957853198051453
Applying scoring rule: InitialNMR (v1.0)
Applying scoring rule: GeneralCRH (v1.0)
Applying scoring rule: LcbCRH (v1.0)
Applying scoring rule: GeneralCRNH (v1.0)
Applying scoring rule: UcbCRNH (v1.0)
Applying scoring rule: NmCRNH (v1.0)
Total ratings: 3579098 post-tombstones and 223999 pre-tombstones
Total ratings created before statuses: 556087, including 486157 post-tombstones and 69930 pre-tombstones.
Total valid ratings: 192160
Unique Raters:  39076
People (Authors or Raters) With Helpfulness Scores:  32863
Raters Included Based on Helpfulness Scores:  26273
Included Raters who have rated at least 1 note in the final dataset:  23496
Number of Ratings Used For 1st Training:  3619999
Number of Ratings for Final Training:  2698602
------------------
Users: 23496, Notes: 69970
initializing notes
initializing users
cpu
epoch 0 0.40559813380241394
TRAIN FIT LOSS:  0.3399314880371094
epoch 50 0.11206676810979843
TRAIN FIT LOSS:  0.08626192808151245
epoch 100 0.11129889637231827
TRAIN FIT LOSS:  0.08425898849964142
Num epochs: 115
epoch 115 0.11129649728536606
TRAIN FIT LOSS:  0.08428216725587845
Global Intercept:  0.1628568470478058
------------------
Re-scoring all notes with extra rating added: {'internalRaterIntercept': None, 'internalRaterFactor1': None, 'helpfulNum': None}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.14149416983127594
TRAIN FIT LOSS:  0.11726519465446472
epoch 50 0.11138982325792313
TRAIN FIT LOSS:  0.0844133123755455
Num epochs: 96
epoch 96 0.11130642890930176
TRAIN FIT LOSS:  0.08429824560880661
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-1', 'raterIndex': 23496, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16229085624217987
TRAIN FIT LOSS:  0.13822472095489502
epoch 50 0.11394057422876358
TRAIN FIT LOSS:  0.08843617886304855
Num epochs: 85
epoch 85 0.11384273320436478
TRAIN FIT LOSS:  0.08817018568515778
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-1', 'raterIndex': 23496, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.1719895601272583
TRAIN FIT LOSS:  0.12891653180122375
epoch 50 0.12944582104682922
TRAIN FIT LOSS:  0.10115989297628403
Num epochs: 84
epoch 84 0.12934477627277374
TRAIN FIT LOSS:  0.10133105516433716
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-2', 'raterIndex': 23497, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.0, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.1564764380455017
TRAIN FIT LOSS:  0.13679614663124084
epoch 50 0.11024019122123718
TRAIN FIT LOSS:  0.08390671759843826
Num epochs: 65
epoch 65 0.11015045642852783
TRAIN FIT LOSS:  0.08361467719078064
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-2', 'raterIndex': 23497, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.0, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.1625971496105194
TRAIN FIT LOSS:  0.1218295469880104
epoch 50 0.12860900163650513
TRAIN FIT LOSS:  0.0998450443148613
Num epochs: 80
epoch 80 0.12854285538196564
TRAIN FIT LOSS:  0.09984190762042999
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-3', 'raterIndex': 23498, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16249951720237732
TRAIN FIT LOSS:  0.1381371021270752
epoch 50 0.11279866099357605
TRAIN FIT LOSS:  0.08706667274236679
Num epochs: 79
epoch 79 0.11270956695079803
TRAIN FIT LOSS:  0.08679283410310745
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-3', 'raterIndex': 23498, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17296093702316284
TRAIN FIT LOSS:  0.12987180054187775
epoch 50 0.1286552995443344
TRAIN FIT LOSS:  0.09953145682811737
Num epochs: 93
epoch 93 0.1285542994737625
TRAIN FIT LOSS:  0.09970027208328247
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-4', 'raterIndex': 23499, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17510266602039337
TRAIN FIT LOSS:  0.1509304940700531
epoch 50 0.13192355632781982
TRAIN FIT LOSS:  0.10659254342317581
Num epochs: 76
epoch 76 0.131842240691185
TRAIN FIT LOSS:  0.10650181025266647
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-4', 'raterIndex': 23499, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16212572157382965
TRAIN FIT LOSS:  0.13665850460529327
epoch 50 0.11367649585008621
TRAIN FIT LOSS:  0.08795454353094101
Num epochs: 79
epoch 79 0.11358115077018738
TRAIN FIT LOSS:  0.08765255659818649
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-5', 'raterIndex': 23500, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.0, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17736336588859558
TRAIN FIT LOSS:  0.1557299792766571
epoch 50 0.13043750822544098
TRAIN FIT LOSS:  0.1049032136797905
Num epochs: 66
epoch 66 0.1303536295890808
TRAIN FIT LOSS:  0.10477226972579956
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-5', 'raterIndex': 23500, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.0, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.15023091435432434
TRAIN FIT LOSS:  0.12547419965267181
epoch 50 0.11003502458333969
TRAIN FIT LOSS:  0.08325959742069244
Num epochs: 97
epoch 97 0.10993592441082001
TRAIN FIT LOSS:  0.0831153616309166
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-6', 'raterIndex': 23501, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17631010711193085
TRAIN FIT LOSS:  0.15190833806991577
epoch 50 0.13062815368175507
TRAIN FIT LOSS:  0.10499828308820724
Num epochs: 88
epoch 88 0.13054625689983368
TRAIN FIT LOSS:  0.10486648976802826
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-6', 'raterIndex': 23501, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16241049766540527
TRAIN FIT LOSS:  0.13594377040863037
epoch 50 0.11258627474308014
TRAIN FIT LOSS:  0.08652327209711075
Num epochs: 79
epoch 79 0.11249891668558121
TRAIN FIT LOSS:  0.08625540137290955
/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/incorrect_filter.py:59: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  ratings_w_user_totals.drop(
/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/incorrect_filter.py:59: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  ratings_w_user_totals.drop(
Applying scoring rule: InitialNMR (v1.0)
Applying scoring rule: GeneralCRH (v1.0)
Applying scoring rule: LcbCRH (v1.0)
Applying scoring rule: GeneralCRNH (v1.0)
Applying scoring rule: UcbCRNH (v1.0)
Applying scoring rule: NmCRNH (v1.0)
Applying scoring rule: GeneralCRHInertia (v1.0)
Applying scoring rule: TagFilter (v1.0)
CRH notes prior to tag filtering: 7349
CRH notes above crhSuperThreshold: 1894
Checking note tags:
notHelpfulOther
  ratio threshold: 0.05310741536536798
notHelpfulIncorrect
  ratio threshold: 0.029573177485621466
notHelpfulSourcesMissingOrUnreliable
  ratio threshold: 0.09141551110962542
notHelpfulOpinionSpeculationOrBias
  ratio threshold: 0.0
notHelpfulMissingKeyPoints
  ratio threshold: 0.10171619204365726
notHelpfulOutdated
  ratio threshold: 0.0
notHelpfulHardToUnderstand
  ratio threshold: 0.05382063690199473
outlier filtering disabled for tag: notHelpfulHardToUnderstand
notHelpfulArgumentativeOrBiased
  ratio threshold: 0.05033946436374779
notHelpfulOffTopic
  ratio threshold: 0.0
notHelpfulSpamHarassmentOrAbuse
  ratio threshold: 0.0002994736018899559
notHelpfulIrrelevantSources
  ratio threshold: 0.03902540974158701
notHelpfulOpinionSpeculation
  ratio threshold: 0.07229388729970718
notHelpfulNoteNotNeeded
  ratio threshold: 0.11037072746701154
outlier filtering disabled for tag: notHelpfulNoteNotNeeded
Total {note, tag} pairs where tag filter logic triggered: 381
Total unique notes impacted by tag filtering: 315
Applying scoring rule: CRHSuperThreshold (v1.0)
Applying scoring rule: ElevatedCRHInertia (v1.0)
Applying scoring rule: FilterIncorrect (v1.0)
Total notes impacted by incorrect filtering: 89
Traceback (most recent call last):
  File "/Users/[REDACTED]/git/communitynotes/sourcecode/main.py", line 96, in <module>
    main()
  File "/Users/[REDACTED]/git/communitynotes/sourcecode/main.py", line 84, in main
    scoredNotes, helpfulnessScores, newStatus, auxNoteInfo = run_scoring(
                                                             ^^^^^^^^^^^^
  File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 469, in run_scoring
    scoredNotes, helpfulnessScores, auxiliaryNoteInfo = _run_scorers(
                                                        ^^^^^^^^^^^^^
  File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 151, in _run_scorers
    modelResultsAndTimes = [
                           ^
  File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 152, in <listcomp>
    _run_scorer_parallelizable(s, ratings, noteStatusHistory, userEnrollment) for s in scorers
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 103, in _run_scorer_parallelizable
    result = ModelResult(*scorer.score(ratings, noteStatusHistory, userEnrollment))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/scorer.py", line 108, in score
    assert set(noteScores.columns) == set(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: all columns must be either dropped or explicitly defined in an output. 
    Extra columns that were in noteScores: {'raterParticipantId_interval', 'raterParticipantId_same'}
    Missing expected columns that should've been in noteScores: set()

Environment

$ python --version                                           
Python 3.11.3

contourpy==1.0.7
cycler==0.11.0
filelock==3.12.0
fonttools==4.39.4
Jinja2==3.1.2
kiwisolver==1.4.4
MarkupSafe==2.1.3
matplotlib==3.7.1
mpmath==1.3.0
networkx==3.1
numpy==1.24.3
packaging==23.1
pandas==2.0.2
Pillow==9.5.0
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2023.3
six==1.16.0
sympy==1.12
torch==2.0.1
typing_extensions==4.6.3
tzdata==2023.3

For example, note that numpy 1.26.2 (specified in requirements.txt) requires python 3.9.

Could you please update the python version in the README to the version of python version you're actually using?

python --version
Python 3.7.9
pip install -r requirements.txt
ERROR: Ignored the following versions that require a different python version: 1.22.0 Requires-Python >=3.8; 1.22.1 Requires-Python >=3.8; 1.22.2 Requires-Python >=3.8; 1.22.3 Requires-Python >=3.8; 1.22.4 Requires-Python >=3.8; 1.23.0 Requires-Python >=3.8; 1.23.0rc1 Requires-Python >=3.8; 1.23.0rc2 Requires-Python >=3.8; 1.23.0rc3 Requires-Python >=3.8; 1.23.1 Requires-Python >=3.8; 1.23.2 Requires-Python >=3.8; 1.23.3 Requires-Python >=3.8; 1.23.4 Requires-Python >=3.8; 1.23.5 Requires-Python >=3.8; 1.24.0 Requires-Python >=3.8; 1.24.0rc1 Requires-Python >=3.8; 1.24.0rc2 Requires-Python >=3.8; 1.24.1 Requires-Python >=3.8; 1.24.2 Requires-Python >=3.8; 1.24.3 Requires-Python >=3.8; 1.24.4 Requires-Python >=3.8; 1.25.0 Requires-Python >=3.9; 1.25.0rc1 Requires-Python >=3.9; 1.25.1 Requires-Python >=3.9; 1.25.2 Requires-Python >=3.9; 1.26.0 Requires-Python <3.13,>=3.9; 1.26.0b1 Requires-Python <3.13,>=3.9; 1.26.0rc1 Requires-Python <3.13,>=3.9; 1.26.1 Requires-Python <3.13,>=3.9; 1.26.2 Requires-Python >=3.9; 1.26.3 Requires-Python >=3.9; 1.26.4 Requires-Python >=3.9
ERROR: Could not find a version that satisfies the requirement numpy==1.26.2 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6)
ERROR: No matching distribution found for numpy==1.26.2

Background

• In the outside of US, there are different political landscapes among nations.
• I have two questions about the notion of "political affiliation" in the following RQ1 in the original BirdWatch paper ( https://arxiv.org/pdf/2210.15723.pdf )
  • RQ1: Can we select a set of Birdwatch notes that both inform understanding (decrease propensity to agree with a potentially misleading claim) and are seen as helpful by a diverse population of users (in particular, users with diverse self-reported political affiliations)? Does algorithmic selection achieve these better than a supermajority voting baseline?

My Questions

• Q1: How do your algorithm be evaluated for non-US nations?

  • In particular,
    • How is party ID of the following form defined in the non-US countries?
    • 
    • e.g. While US and UK has the two party system, many EU nations or Asian nations like Korea or Japan have many parties in their legislative branch of the government.

• Q2: Could we increase the robustness of the bridging feature and diversity by the following selection methods of CN-raters at the preview phase at which only contributors could view and rate the proposed notes.

  • The methods:
    • 1. Build a classifier model to predict party-ID for given input user's post's(tweet's) texts to prevent lies on their true political affiliations.
    • 2. For each predicted party-ID label, select N*K users, where K is the number of party-IDs, where N is an arbitrary constant integer.
    • 3. Expose given proposed note to only the N*K users and evaluate it.
  • The expected behavior of this method: we would obtain the similar results with the following three figures in the original paper.

Describe the bug
notes are not always shown in case of user not logged in

To Reproduce
Steps to reproduce the behavior:
logout from twitter - visit a tweet which has a note - like this one:
https://twitter.com/POTUS/status/1637136790208356354
but here is another tweet which has 2 notes, and only 1 is shown
https://twitter.com/elonmusk/status/1651602599345373186

Expected behavior
notes should be always visible for everyone for example if tweet was shared with someone who does not have an account or a tweet which was embedded.

Screenshots

Environment
any web browser desktop or mobile

Describe the bug
Elon told me to open this issue. Where are the unit tests to verify that results are as expected? They would be extremely useful, especially after refactors like the recent weighting adjustments.

Expected behavior
There should be unit tests to guard against regressions and document, in code, expected behaviors.

Hi.

I just read the Birdwatch white-paper which Community Notes is based on. I realized that in the Appendix, specifically table 10 which relates to RQ2, the model for Tweet agreement on a 5-point scale has limited explanatory power. Modestly speaking.

Using an unweighted OLS with an adjusted R-squared of 0.03 but with statistically significant variables (few of them) and a statistically significant F-statistics just means the model is "better" than nothing. In the lens of statistics this model is useless for explainability or to use the estimated dependent variable 'Tweet agreement' in a bivariate association as erroneous.

The percentage of standard deviation explained is:
1 - sqrt(1 - r_squared)*100 = 2.02%

In summary, the model can explain about 4% in the variability of Tweet agreement and about 2% of the standard deviation of its errors.

Have you thought about redoing the analysis/paper with statistically rigorous methods?

Kindly

I'd like to hydrate community notes based on noteId, just like you can hydrate tweets with their tweetId. However, I'm unable to find a way to do this with the twitter API, and it looks like the actual content of the community note is not present in the released data.

How can I get the body of the actual community note?

Here is the X/Twitter API v2 endpoint for a tweet: https://api.twitter.com/2/tweets/{tweetId}. However, it does not work when I put in a noteId.

Additionally, I see that the url twitter.com/i/birdwatch/n/{nodeId} goes to the community note, but does not seem (as far as I'm aware) to be accessible to the API.

Is your feature request related to a problem? Please describe.

On some controversial subjects, authors who are faced with notes that contradict them simply delete their tweet and republish it afresh.

Even when adding new notes, they continue.

I'd like to open the discussion to see how this kind of behavior could be handled. Sometimes several notes that provided interesting context have disappeared.

Describe the solution you'd like

I don't have any particular solution, but I'd like to have a discussion to find something simple to implement.

If a tweet is deleted, would it be possible to find the note associated with it?

If a note disappears, could its author be notified (and still have access to his copy)?

One idea might be to mention in the attributes that the original tweet has been deleted (with a screenshot of the original containing the presence of notes) if an author of the original note comes back to the new tweet. (I know it's a complicated process).

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context

It's a complex subject so discussion would help gather ideas to sort through.

Is your feature request related to a problem? Please describe.

When a post you've interacted with (replied to or liked) gets Note'd, you receive a notification afterward to inform you. However, when you click the notification, the only easily accessible options are "rate" instead of the actions you'd prefer after discovering someone is likely a liar or unreliable, which are "unfollow," "block," or "mute."

Describe the solution you'd like

Change the UI when you click the notification to five ( max ) options along with the Note.

Rate , Unfollow , Block , Mute and Uninteract ( which removes any likes or repost you've done of that )

Detailed options to rate only come up after you select the rate option.

Describe alternatives you've considered
The alternative is to click the bad post ( giving it more views and interaction ) and then go to options and carry out the task. It's not very intuitive. Especially the part about clicking the post above the note to open it since it's greyed out.

Additional context

This also helps stop drive by downratings from the very people who are the most likely to have fallen for the post. It also organically punishes bad actors by making it easier for people to unfollow or block them.

Is your feature request related to a problem? Please describe.
Today X does notify readers if a note was added to a post to which they replied, liked or reposted. Now picture the other way around: thousands of readers may have seen a post with a Note rated helpful (e.g. in first hours of the tweet), but that Note still can get rejected in the following hours (per footnote here). The problem is that these initial readers have context which was de-ranked, and are not made aware of it.

Describe the solution you'd like
A similar notification in the case the accepted Note changes and/or is deleted. That way users who interacted with the initial post are at least aware of the update and receive (eventually) the highest rated context.

Describe alternatives you've considered
Good question. I guess perhaps a reply-tweet from a Community account, that explicitly calls out the modification on what was so far the active note (and which, I suppose, may appear on the timeline of folks who engaged with the ranked tweet at first).

Additional context
X conversation here (originally in French): https://twitter.com/nicoallgood/status/1694718054951997831?s=20 . Tweet quoted in the top tweet has very high engagement, and initially had a note that was then removed the next day.

Community Member A makes an original note A on a post

Community Member B makes a "note not needed" note B on the same post, making his case why the note A should be voted down.

Community Member C or original member A wants to make a counterpoint to B ( let this counterpoint be called C ), stating why the note is needed or why the counterpoint B is false/irrelevant.

Currently there's no proper place to post Note C. If you post it in the Notes section, people might vote to make it visible. But what you're actually arguing for is to keep A visible. So all votes on my note C should be going to note A that I quoted.

Feature:

This could be solved with a reply option for Notes, atleast NNN notes, that shows those replies under the NNN note.

Hi! Amazing work 👏

Describe the bug
The requirements.txt file is missing the scipy dependency.
In README.md it states:

While your normal Python install may "just work" if you're lucky, if you run into any issues and want to install the exact versions of Python packages that we've tested the code with, please create a new virtual environment and install the packages from requirements.txt:

python -m venv communitynotes_env
source communitynotes_env/bin/activate
pip install -r requirements.txt

Then after downloading the data files (see next section) into /sourcecode/, you will be able to run:

cd sourcecode
python main.py

Doing this will result in:

  File "../communitynotes/sourcecode/scoring/reputation_matrix_factorization/weighted_loss.py", line 1, in <module>
    import scipy.sparse as sps
ModuleNotFoundError: No module named 'scipy'

This can be easily fixed by adding the dependency in requirements.txt

To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/twitter/communitynotes.git
2. conda create --name communitynotes python=3.7.9 (or use venv)
3. conda activate communitynotes (or use venv)
4. pip install -r requirements.txt
5. Download data from https://twitter.com/i/communitynotes/download-data into /sourcecode/data folder
6. python main.py --enrollment data/userEnrollment-00000.tsv --notes data/notes-00000.tsv --ratings data/ratings-00000.tsv --status data/noteStatusHistory-00000.tsv --outdir data

I've unfortunately lost the full output logs, but this line divided by zero and caused the script to fail:

https://github.com/twitter/communitynotes/blame/d3f0ea19b55fac35a0bac0bda7c33d73425fed5e/sourcecode/scoring/note_status_history.py#L181C43-L181C54

Specifically, len(oldCrhNotes) was zero. I'll try to find time to debug this later, but opening this issue in case this is obvious to someone with more context (@bradmiller ?). Noting that this with the most recent (as of today) data files and python 3.7.9.

Is your feature request related to a problem? Please describe.
Hey! I really like the concept of Community Notes and I appreciate this is open-source. However, there doesn't currently appear to be a way to verify that the code in this repository is the code that is actually running on the web experience. This classifies as a feature request, but could also be considered a security issue as well.

Describe the solution you'd like
A way for any user to technically verify that the code in releases made to mobile and web application for Community Notes is the same code running in this repository.

Thanks for any information!

https://developer.kevin.eu/home/mobile-sdk/ios/account-linking @jdm-twtr

Hello!

Thanks for sharing this code with the community!
However, as I've been trying to understand the algorithm, I've become a bit confused. It seems like the Matrix Factorization model for Community Notes only uses a single latent variable, which means there's only one feature to model the diversity of notes and users.

This must make it difficult to capture the full range of differences between people. With only one feature per person, how can you hope to capture things like race, gender, preferences, age, political views, cultural background, and so on?

Intuitively, with just one variable, the only information you can use is whether a rater would typically agree or disagree with the average. The Wikipedia article on Matrix Factorizations seems to confirm this, stating that a matrix factorization with one latent factor is equivalent to a top popular recommender (e.g., one that recommends the items with the most interactions without personalization).

I understand that significant regularization is applied to the note intercept term, which means any notes with a high intercept term must have been accepted by both classes of people. However, I wonder how this approach compares to simply using average helpfulness scores along with the variance in ratings.

This way, you could show notes that are both on average helpful and non-polarizing (low variance) (as long as enough people saw the note). I think this would produce similar results as the current algorithm, but it would be easier to understand and compute. What are your thoughts on this?

Finally, I'm curious about how to assess the performance of the algorithm. How do we know that the notes are actually helpful for an actually diverse group of people? This is crucial information if want to try to improve the algorithm.

Describe the bug
Account stays permanently locked out from writing notes, even if requirements to unlock are met.

To Reproduce
Steps to reproduce the behavior:

1. Get locked out from writing by getting 3 of the last 5 rated notes deemed "not helpful";
2. After getting locked out, get all 3 notes previously rated as "not helpful" to "needs more ratings";
3. After that, the locked out message dissapears for around one hour, returning later (even with 0 notes rated as "not helpful");

• Even if you increase your Rating Impact by 5 (as required to unlock the account) the step 3 occurs again.

Expected behavior
The account ability to write notes should be unlocked after the requirements are met.

Screenshots

Environment
Mobile (Samsung S20 - Android 11; SM-G980F Build/RP1A.200720.012)
Web (Chrome 115.0.5790.168)

Additional context
I already increased my Rating Impact score by 14 since getting this bug, but after the message appears later it keeps asking for 5 more.

I'm having this problem for around 24 hours.
For contact in X/Twitter: @bananajoga

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Describe the bug
I attempted to run sourcecode/main.py and received the following error:

AttributeError: module 'numpy' has no attribute 'object'.np.objectwas a deprecated alias for the builtinobject. To avoid this error in existing code, use objectby itself. Doing this will not modify any behavior and is safe. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'object_'?

If I replace instances of 'np.object' with 'object', the error is resolved, but I also have to do the same with 'np.str' and 'np.float'.

This is a Windows environment. Is there a suggested environment? I did not see any in the documentation.

I saw this was referenced in issue #87, so I also tried to run on Ubuntu 22.04.2 LTS (via WSL2) and received the same error. The comments on the issue said the Python script was meant to run in *nix environments, but it's not clear which.

To Reproduce
Steps to reproduce the behavior:

1. With data files present in the 'sourcecode' folder and command prompt at the same folder, run:
   python .\main.py --enrollment .\userEnrollment-00000.tsv --notes .\notes-00000.tsv --ratings .\ratings-00000.tsv --status .\noteStatusHistory-00000.tsv --outdir scored_notes.tsv
2. Observe output for errors.

Expected behavior
The script should execute without error.

Environment
Windows 11 Pro (22621.1702)
Python 3.11.3
numpy 1.24.3

Also:
Ubuntu 22.04.2 LTS (via WSL2)
Python 3.10.6
numpy 1.24.3

Is any developer other than Twitter successfully running an instance Community Notes? It's a great product and could help the rest of the Internet.

"Note Stepping" ( as it is called within the Community) is when someone puts up a bogus note as a way to suppress votes on another note that they disagree with, regardless of it's validity.

The problem arises from the fact that notes are arranged by time of submission, with the newest note showing at the top. And raters are most likely to see the note at the top, and less likely to scroll down and see those below it, especially when the top note is a bogus one.

So all someone has to do, if they see a pending note they don't like is to submit another note, and the time it takes for the original note to become visible will get much longer ( if not impossible ) since it will stop receiving as many ratings.

Fix:

Just change the way pending notes are sorted in the pending notes area. Instead of sorting by new, either sort by random or sort by top. Best is to sort by random at first and then sort by top once a certain threshold has been exceeded.

Describe the bug
We were supposed to get a "heads up" if a tweet we liked, replied to or retweeted gets a community note added to it.
https://twitter.com/CommunityNotes/status/1628158167006994436

I saw the below tweet long before the community note was added to it. I was concerned with how long it was taking for a community note to be added so I liked it to test if people who interacted with the tweet would actually be notified.

You can see my like still there, and I was never notified.

To Reproduce
Steps to reproduce the behavior:

1. Like a tweet before the fact check
2. Tweet gets fact checked

Expected behavior
I would expect some form of notification saying a tweet I interreacted with was given a community note.

Screenshots

Environment
I've check on both android and web no notification was sent.

Found a problem with the Community notes account. The account exists, it is shown as mine, the ratings and notes written are visible, but accesing the settings results in an error. Notes can still be rated. Shortcuts to my account are missing. And the page asks to joing the community notes again.

Steps to reproduce the behavior:

1. Click the button to rate a note, shortcuts to the related CN account are missing. The page suggests to join instead.
2. Accesing the account from a link, everything seems OK but the settings page returns an error "this page doesn't exist".

Firefox/Android 13, web version

Describe the bug

Rate it button appears for a CN which I've already rated.

Expected behavior

I should not be asked to Rate it if I've rated it.
I can't always remember if I've already rated a particular note or not, especially since some posts may have several notes.

Screenshots

I get invited to rate a note:

But when I click on the button I get told I've already rated the CN. In this case there are additional notes, but maybe I don't want to rated additional notes, or there may be no additional notes.

Environment

Firefox (latest)

While there are some relevant and good quality Notes that don't require anything other than text, the vast majority of notes (>90%) that are short and do not contain any links are either bogus, express personal opinions, have bad grammer and spelling, or are harassment/abuse.

It's appropriate for those to have a higher threshold to become visible.

Is your feature request related to a problem? Please describe.

When there is a hot topic with a lot of misinformation going around, it is frustrating to have to write notes on every post and wait for all the notes to slowly reach helpful status.

Describe the solution you'd like

Be able to use the note ID to reuse a high-quality already helpful note and have that note appear faster than usual.

To encourage people to reuse notes, and also to encourage people to write high-quality notes that other people might consider reusing, it would be necessary to give writing impact to both the user who reused it and the user who created the original note.

Describe the bug
When the writing impact was decreased and the ability to write notes are restricted, sometimes it will end up in the profile "Unlock your ability to write" section showing an message like

Increase your Rating Impact to 5, from your current score of 13.

This message makes no sense since you can't increase a big number to a smaller one.

To Reproduce
Steps to reproduce the behavior:

1. Rating notes so the Rating Impact is high enough (13 in this case).
2. Writing notes but get disapproved so the writing impact is negative .
3. Up to a certain point, the writing ability will be restricted (3 out of 5 notes rated unhelpful).
4. Go to profile, check the section mentioned above.

Expected behavior
A clear, accelerate, and achievable advice to unlock the writing ability.

Screenshots

Environment
Twitter Web on Windows 11, date is 22/06/2023

Hi there,

The sign up button (https://mobile.twitter.com/i/communitynotes/about) seems to be hidden below the scrollable area on iOS Safari. In both portrait and landscape orientation.
Is this intentional?

iPhone SE 16.1.1

The Discussions feature is currently disabled for this repo.

I think that enabling it can benefit the development and contributions to this repo.

Description
As we are working to empower people on Twitter to add notes to tweets that might be misleading, it is important to have a clear understanding of what each of the 12 Python files in the sourcecode/ directory does. While the explanation of the codes is written on themselves, having a bird's eye view in layperson's terms through a README.md or README.txt would greatly help people who want to participate in a more informed world.

Proposed Changes
Add a README.md or README.txt file in the sourcecode/ directory to give an overview of the purpose of each of the 12 Python files. The README should explain in layman's terms what each file does, and how they work together to power Community Notes.

Benefits
A clearer understanding of the source code for people who are interested in participating in the project. Increased transparency and accessibility of the project, making it easier for people to contribute.

Impact
A well-written README will help the Community Notes project to grow and improve, by providing a clear and concise overview of the code that powers the project. This will make it easier for people to understand how the project works and how they can contribute to it.

Checklist

• Add the README.md or README.txt file to the sourcecode/ directory.
• Verify that the README.md or README.txt file accurately explains the purpose of each of the 12 Python files in layperson's terms.
• Verify that the file is well-written, easy to understand, and accurately represents the project.

Additional Information:
Dear Twitter developers,

I hope this message finds you well. I wanted to bring to your attention a suggestion that could enhance the usability and accessibility of the Community Notes project. As you are the main contributors and have a deep understanding of the Python codes powering the project, I believe you would be the best suited to address this issue.

Thank you, and I look forward to your response.

Best regards,
Ralph

I am getting stuck in the 1/3 step. Can you guys let us know the reason, and what might the step we should be taking to resolve this?

Is your feature request related to a problem? Please describe.
Community notes is a very useful feature and now it is extended on media context - would be nice to somehow be extended on user accounts as well.
lots of twitter account created for a purpose of spread misleading information. Those accounts own several community notes on their tweets. The problem is that these accounts can create new tweets without indicating to readers that this account became controversial.

Describe the solution you'd like
Would be nice to highlight for readers that this account has many community noted tweets.

Describe alternatives you've considered

• Restrict reachability if an account reaches a certain level of community noted tweets
• show a label on user account like a number which would reflect on the number of community noted tweets
• give a community notes label to user account by enabling community notes on user account (not just on tweets) - so if a user account would get a community note then label the account with a visible sign for every tweets the user creates.

It would be a better UX to default to showing notes in the language of the device and even better to allow the user to select their preference order for notes and best to expose notes in all available languages to the user.

Hi,

Now that there are sufficiently many ratings to split the rating data into two TSVs, how do we run main.py while passing in multiple rating filepaths as command line args? Or is the expected usage to combine the two files into one and then pass that in?

Thanks,
Raj

Is your feature request related to a problem? Please describe.
I signed up for Community Notes but I find that the content I am exposed to can be unpleasant and hurtful. Also, as a sometime Wikipedian, I thought it might be interesting or even fun. So far though, I've had no impact and engaging with flat-earth or other intentionally misleading nonsense is a waste of my time. I want out!

I haven't found a link, button etc to remove myself from the program.

Describe the solution you'd like
A way to quit the program, with an option to explain why.

Describe alternatives you've considered
The only alternative would be to stop using Twitter...

[update: I'm using Twitter less]

And, I can't even DM @lucasnantonio as I'm not verified!

Additional context

The infrastructure algorithm seems to have the wrong amount of trust in people; communitynotes is failing on a note with 4.8M Views with demonstrably false health re. a public health emergency, violating https://help.twitter.com/en/rules-and-policies/crisis-misinformation (the violated policy). It's failing on note https://twitter.com/i/birdwatch/n/1648714744034123776 based on direct, 100% definitive factual, web-based evidence, despite multiple Reports.

I reported this via https://help.twitter.com/en/forms/community-note?note_id=1648714744034123776, as well as by sending a DM to @communitynotes, marked "URGENT!" and tweeting to @communitynotes:

FALSE INFO IN YOUR NOTE: "The bivalent Pfizer vaccine is still approved by the FDA."

NO!

The BIVALENT is ONLY AUTHORIZED under EUA.

It IS NOT FDA APPROVED!!! ONLY MONOVALENT IS/WAS! "APPROVED" is NOT CORRECT!…".
To Reproduce
Steps to reproduce the behavior:

1. Go to https://twitter.com/sneako/status/1648696595439927305
2. Scroll down to the second (globally visible) community note.
3. See "The bivalent Pfizer vaccine is still approved by the FDA." at the end of the note.
4. Go to https://help.twitter.com/en/rules-and-policies/crisis-misinformation
5. See "advance a claim of fact, expressed in definitive terms;
   be demonstrably false or misleading, based on widely available, authoritative sources; and
   be likely to impact public safety or cause serious harm."
6. Go to https://eua.modernatx.com/recipients .
7. See "Moderna COVID-19 Vaccine, Bivalent has not been approved or licensed by the FDA"
8. Go to https://www.pfizer.com/products/product-detail/pfizer-biontech-covid-19-vaccine .
9. See "Pfizer-BioNTech COVID-19 Vaccine, Bivalent has not been approved or licensed by FDA"
10. (optional) Rate note for

• Sources do not support note (Doesn't say it's approved; the word "approved" does not even appear in the source.)
• Incorrect information: FALSE: "The bivalent Pfizer vaccine is still approved by the FDA."
• Typos or unclear language ("monovalant")
• Misses key points or irrelevant (Misses distinction between FDA approved and FDA EUA'd.)
• Other (There's no debate over whether "The bivalent Pfizer vaccine is still approved by the FDA." No one disputes it. It's readily verifiable to 100% certainty that it's false.)

11. (optional) DM @communitynotes, tweet to @communitynotes and @TwitterSafety about it.
12. Report (# 0322519944.
13. Get told "we didn’t find a violation of our rules. "
14. Re-report. # 0328844553. Ditto, and ditto with # 0320612475 (May 8) and # 0317707377 (April 20) too.

Expected behavior

1. Reports of @community notes with of false info in violation of policy should be actioned. Algorithm should be tweaked so that action will be taken if this note is reported yet again.
2. I've noticed that some @communitynotes accounts seem to be (based on volume) role accounts run by paid writers. Twitter/the algorithm should detect and Twitter should investigate. Money should not define truth of @communitynotes.

Informed consent requires those vaccinated be aware if an injection is FDA approved or FDA EUA'd, to balance benefits and risks, especially as there are fully tested FDA-approved preventatives.

Describe the bug
The published note scoring code is incompatible with the current data file format.

To Reproduce

1. Download source code
2. Download Birdwatch data files
3. Run python main.py
4. Fails assertion while processing user enrollment status data

Expected behavior
Program should output scored notes

Screenshots

❯ python main.py
invalid literal for int() with base 10: 'noteId'
invalid literal for int() with base 10: 'noteId'
invalid literal for int() with base 10: 'noteId'
invalid literal for int() with base 10: 'timestampOfLastStateChange'
Traceback (most recent call last):
  File "main.py", line 53, in <module>
    run_scoring()
  File "main.py", line 44, in run_scoring
    _, ratings, noteStatusHistory, userEnrollment = process_data.get_data(
  File "process_data.py", line 32, in get_data
    notes, ratings, noteStatusHistory, userEnrollment = read_from_tsv(
  File "process_data.py", line 122, in read_from_tsv
    assert len(userEnrollment.columns.values) == len(c.userEnrollmentTSVColumns) and all(
AssertionError: userEnrollment columns don't match:
['timestampOfLastEarnOut'] are extra columns,
[] are missing.

Environment

❯ python --version
Python 3.10.9

❯ pip freeze
contourpy==1.0.7
cycler==0.11.0
fonttools==4.38.0
kiwisolver==1.4.4
matplotlib==3.6.2
numpy==1.23.5
packaging==23.0
pandas==1.5.1
Pillow==9.4.0
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2022.7.1
six==1.16.0
torch==1.13.0
typing_extensions==4.4.0