sillsdev / chorus Goto Github PK

View Code? Open in Web Editor NEW

5.0 14.0 26.0 59.73 MB

End-user collaboration library via 3-way xml merging and hg dvcs under the hood

Shell 0.05% C# 95.07% CSS 0.04% XSLT 4.57% HTML 0.16% Batchfile 0.12%

chorus synchronization conflict xml merge send-receive dvcs hg fieldworks hacktoberfest

chorus's People

Contributors

Stargazers

Watchers

chorus's Issues

Can't run Chorus unless included Mercurial is used

It's not possible to use the system provided mercurial. HgRepository.GetMercurialConfigInMercurialFolder() crashes if MercurialLocation.PathToMercurialFolder is not set which is the case if chorus can't find a Mercurial folder.

Increase 1-megabyte file size limit

The maximum file size allowable in repositories that Chorus handles is 1 megabyte, for reasons discussed here. That commit was made in 2011. Many of those reasons still apply (we don't want to let people shoot their repositories in the foot by adding hour-long video files, for example), but the 1-megabyte limit is becoming too small. Most people have phones with cameras that take 3- or 4-megabyte photos, and if they want to add a photo to their repository, they have to learn how to trim its size and reduce its JPG quality. CPUs and network speeds have also increased, on average, over the past decade, so that larger file sizes, such as 5 or 10 megabytes, probably won't harm repositories all that much now.

So while we still don't want to allow adding video files to Chorus repositories, I believe it's time to consider increasing the file size. A limit of 5 megabytes would allow photos taken by most low- to mid-grade consumer cameras these days, but many high-grade phones are taking photos that are even larger: 6 or 8 megabytes. So I propose increasing the file size limit from 1 megabyte to 10 megabytes. That will allow photos taken by most cameras to be added to a repository. It would also mean that .wav files longer than 5.6 seconds would be possible as well: a .wav file would have to reach 56.69 seconds before its size passed the 10-megabyte limit.

dotnet no longer supports thread abort

if you call Thread abort in any new dotnet code it will throw an exception. This is a problem in Chorus because it's used in the HgProcess output reader here:

chorus/src/LibChorus/Utilities/HgProcessOutputReader.cs

Lines 87 to 90 in 04e1e28

 if (_outputReader != null) 

 _outputReader.Abort(); 

 if (_errorReader != null) 

 _errorReader.Abort();

Ultimately the code each of those threads is running needs to be rewritten to check something like a cancellation token which we could cancel there instead of aborting the threads.

ChorusHub as separate Linux package

I'm submitting this here, since the following message on sil.org was posted, but the contact page is broken: If you would be interested in a separate ChorusHub package, please contact us.

I'm wanting to run Chorus Hub on a Linux-based file/media server, but I don't need the whole fieldworks package. Is a separate package available? It would need to be installable by a non-specialist.

Update included Mercurial

We're using an old version of Mercurial. We should update this to a current version.

Implement Git as another VCS backend

This depends on #192.

Password Encryption/Decryption will crash LibChorus on Linux .NET6

ServerSettingsModel has the methods EncryptPassword and DecryptPassword. These use System.Security.Cryptography.ProtectedData, which compiles for .NET6 but throws a runtime exception on Linux.

var encryptedData = ProtectedData.Protect(Encoding.Unicode.GetBytes(encryptMe),
		Encoding.Unicode.GetBytes(EntropyValue), DataProtectionScope.CurrentUser);
return Convert.ToBase64String(encryptedData);

No one is currently using this, but if they try it will break. We need a cross-platform password solution, or one that at least adapts to platform-specific calls.

BinaryFormatter introduces security vulerabilities and is disabled in dotnet 8

dotnet 8 disables BinaryFormatter for good reason, it's very insecure.

HgResumeTransport is using it here and a couple lines down in ReadServerRevisionCache. From what I can tell it's just being used to persist a fairly simple data structure to the file system, there's even some notes about fixing it. That time has come.

What's this file used for and can we just change the format and delete the old version? or must we migrate the data?

also, we can't just kick this down the road anymore. dotnet 9 will remove the API altogether

Outdated Mercurial (version 3.0.1) causes "SSL: TLSV1_ALERT_PROTOCOL_VERSION" in Linux Flatpak install

Hi,
I have been using FieldWork Language Explorer for some time on an Archlinux machine, first via a windows virtual machine, and now with a Flatpak installation on Archlinux. I also have my own mercurial installation on a server of mine, but I am having issues with the send/receive function of FLEx, that is implemented - unless I'm very much mistaken - through chorus, hence me posting here (please let me know if this is not the right way to report this).

I have the following output on failure to sync:

Syncing...
Started at 2024-01-30 15:46:55Z
Local User: mark
LanguageForge User: flex
Repository URI: http://hg.archraspberrypi.tk/fieldwork_language_explorer
Local Directory: [PROJECT-PATH]/tungag

Executing: recover 
standerr: no interrupted transaction available

Executing: addremove -s 100 -I **.wav
Executing: parents --template "changeset:{rev}:{node|short}
branch:{branches}
user:{author}
date:{date|rfc822date}
tag:{tags}
summary:{desc}
parent:{p1rev}:{p1node}"
standout: changeset:126:dc2ebed055e6
branch:7500002.7000072
user:mark
date:Sun, 21 Jan 2024 13:17:20 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync
parent:125:021330251315b7e38bdec4a8972b39a86660e079

changeset:126:dc2ebed055e6
branch:7500002.7000072
user:mark
date:Sun, 21 Jan 2024 13:17:20 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync
parent:125:021330251315b7e38bdec4a8972b39a86660e079
Executing and caching: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Executing and caching: log -r126 --template "{node}" 
standout: dc2ebed055e6a4760ead4036486d890eeda7dc74

Using cached result: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Executing and caching: log -r125 --template "{node}" 
standout: 021330251315b7e38bdec4a8972b39a86660e079

Executing: parents --template "changeset:{rev}:{node|short}
branch:{branches}
user:{author}
date:{date|rfc822date}
tag:{tags}
summary:{desc}
parent:{p1rev}:{p1node}"
standout: changeset:126:dc2ebed055e6
branch:7500002.7000072
user:mark
date:Sun, 21 Jan 2024 13:17:20 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync
parent:125:021330251315b7e38bdec4a8972b39a86660e079

changeset:126:dc2ebed055e6
branch:7500002.7000072
user:mark
date:Sun, 21 Jan 2024 13:17:20 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync
parent:125:021330251315b7e38bdec4a8972b39a86660e079
Executing and caching: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Executing and caching: log -r126 --template "{node}" 
standout: dc2ebed055e6a4760ead4036486d890eeda7dc74

Using cached result: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Executing and caching: log -r125 --template "{node}" 
standout: 021330251315b7e38bdec4a8972b39a86660e079

Storing changes in local repository...
Split up project file: tungag.fwdata
Writing the linguistics data....
Writing reversal data....
Writing morphology and syntax data....
Writing lexical data....
Writing text corpus data....
Writing wordform and punctuation data....
Writing discourse data....
Writing phonology data....
Writing the anthropology data....
Writing the other data....
Writing the general data....
Writing user-defined list data....
Writing language project data....
Copying settings files...
Finished splitting up project file: tungag.fwdata
Executing: status  -mau  -X "~~*.txt" -X "**.NewChorusNotes" -X "**.fwdata" -X "**.fwdata-replaced" -X "**.fwdata-x" -X "**.fwdb" -X "**.bad" -X "**.bak" -X "**.flextext" -X "**.fwbackup" -X "**.fwstub" -X "**.lint" -X "**.log" -X "**.orig" -X "**.oxekt" -X "**.oxes" -X "**.oxesa" -X "**.tmp" -X "**.xml" -X "**.zip" -X "**.fwdata.lock" -X "**.dupid" -X "Temp/**.*" -X "BackupSettings/**.*" -X "WritingSystemStore/trash/**.*" -X "WritingSystemStore/WritingSystemsToIgnore.xml.ChorusNotes" -X "OtherRepositories/**.*" -X "**.mpa" -X "**.mpe" -X "**.mpg" -X "**.mpeg" -X "**.mpv2" -X "**.mp2" -X "**.mp4" -X "**.mov" -X "**.wmv" -X "**.rm" -X "**.avi" -X "**.wvx" -X "**.m1v" 
standout: M CachedSettings/WritingSystemStore/lcm.ldml
M General/LanguageProject.langproj
M Linguistics/Lexicon/Lexicon_03.lexdb
M WritingSystemStore/lcm.ldml
? CachedSettings/ConfigurationSettings/Dictionary/Preview.css
? CachedSettings/SharedSettings/Mark.ulsx
? CachedSettings/SharedSettings/mark.ulsx
? ConfigurationSettings/Dictionary/Hybrid.fwdictconfig
? ConfigurationSettings/Dictionary/Lexeme.fwdictconfig
? ConfigurationSettings/Dictionary/Preview.css
? ConfigurationSettings/Dictionary/Root.fwdictconfig
? ConfigurationSettings/LexEntry.fwlayout
? ConfigurationSettings/ReversalIndex/en.fwdictconfig
? ConfigurationSettings/ReversalIndex/tpi.fwdictconfig
? SharedSettings/LexiconSettings.plsx
? SharedSettings/Mark.ulsx
? SharedSettings/mark.ulsx
? WritingSystemStore/lcm-fonipa.ldml

Executing: status -d 
Adding files to be tracked ( -I "[PROJECT-PATH]/tungag/**.ChorusNotes" -I "[PROJECT-PATH]/tungag/FLExProject.ModelVersion" -I "[PROJECT-PATH]/tungag/FLExProject.CustomProperties" -I "[PROJECT-PATH]/tungag/do_not_share_project.txt" -I "[PROJECT-PATH]/tungag/.hgignore" -I "[PROJECT-PATH]/tungag/**.list" -I "[PROJECT-PATH]/tungag/**.style" -I "[PROJECT-PATH]/tungag/LinkedFiles/AudioVisual/**.*" -I "[PROJECT-PATH]/tungag/LinkedFiles/Others/**.*" -I "[PROJECT-PATH]/tungag/LinkedFiles/Pictures/**.*" -I "[PROJECT-PATH]/tungag/SupportingFiles/**.*" -I "[PROJECT-PATH]/tungag/CachedSettings/ConfigurationSettings/*.fwlayout" -I "[PROJECT-PATH]/tungag/CachedSettings/ConfigurationSettings/**.fwdictconfig" -I "[PROJECT-PATH]/tungag/CachedSettings/ConfigurationSettings/**Overrides.css" -I "[PROJECT-PATH]/tungag/CachedSettings/WritingSystemStore/*.ldml" -I "[PROJECT-PATH]/tungag/CachedSettings/SharedSettings/*.plsx" -I "[PROJECT-PATH]/tungag/Linguistics/Reversals/**.reversal" -I "[PROJECT-PATH]/tungag/Linguistics/Lexicon/*.lexdb" -I "[PROJECT-PATH]/tungag/Linguistics/TextCorpus/*.textincorpus" -I "[PROJECT-PATH]/tungag/Linguistics/Inventory/*.inventory" -I "[PROJECT-PATH]/tungag/Linguistics/Discourse/Charting.discourse" -I "[PROJECT-PATH]/tungag/Linguistics/**.featsys" -I "[PROJECT-PATH]/tungag/Linguistics/MorphologyAndSyntax/AnalyzingAgents.agents" -I "[PROJECT-PATH]/tungag/Linguistics/MorphologyAndSyntax/MorphAndSynData.morphdata" -I "[PROJECT-PATH]/tungag/Linguistics/Phonology/PhonologicalData.phondata" -I "[PROJECT-PATH]/tungag/Anthropology/DataNotebook.ntbk" -I "[PROJECT-PATH]/tungag/Other/ReferenceSystem.srs" -I "[PROJECT-PATH]/tungag/Other/*.ArchivedDraft" -I "[PROJECT-PATH]/tungag/Other/Translations.trans" -I "[PROJECT-PATH]/tungag/Other/Settings.ImportSetting" -I "[PROJECT-PATH]/tungag/Other/Books/*.bookannotations" -I "[PROJECT-PATH]/tungag/Other/Books/*.book" -I "[PROJECT-PATH]/tungag/General/FLExFilters.filter" -I "[PROJECT-PATH]/tungag/General/FLExAnnotations.annotation" -I "[PROJECT-PATH]/tungag/General/LanguageProject.langproj" -I "[PROJECT-PATH]/tungag/General/UnownedPictures.pictures" -I "[PROJECT-PATH]/tungag/General/VirtualOrdering.orderings" -I "[PROJECT-PATH]/tungag/**.ChorusRescuedFile" -X "[PROJECT-PATH]/tungag/~~*.txt" -X "[PROJECT-PATH]/tungag/**.NewChorusNotes" -X "[PROJECT-PATH]/tungag/**.fwdata" -X "[PROJECT-PATH]/tungag/**.fwdata-replaced" -X "[PROJECT-PATH]/tungag/**.fwdata-x" -X "[PROJECT-PATH]/tungag/**.fwdb" -X "[PROJECT-PATH]/tungag/**.bad" -X "[PROJECT-PATH]/tungag/**.bak" -X "[PROJECT-PATH]/tungag/**.flextext" -X "[PROJECT-PATH]/tungag/**.fwbackup" -X "[PROJECT-PATH]/tungag/**.fwstub" -X "[PROJECT-PATH]/tungag/**.lint" -X "[PROJECT-PATH]/tungag/**.log" -X "[PROJECT-PATH]/tungag/**.orig" -X "[PROJECT-PATH]/tungag/**.oxekt" -X "[PROJECT-PATH]/tungag/**.oxes" -X "[PROJECT-PATH]/tungag/**.oxesa" -X "[PROJECT-PATH]/tungag/**.tmp" -X "[PROJECT-PATH]/tungag/**.xml" -X "[PROJECT-PATH]/tungag/**.zip" -X "[PROJECT-PATH]/tungag/**.fwdata.lock" -X "[PROJECT-PATH]/tungag/**.dupid" -X "[PROJECT-PATH]/tungag/Temp/**.*" -X "[PROJECT-PATH]/tungag/BackupSettings/**.*" -X "[PROJECT-PATH]/tungag/WritingSystemStore/trash/**.*" -X "[PROJECT-PATH]/tungag/WritingSystemStore/WritingSystemsToIgnore.xml.ChorusNotes" -X "[PROJECT-PATH]/tungag/OtherRepositories/**.*" -X "[PROJECT-PATH]/tungag/**.mpa" -X "[PROJECT-PATH]/tungag/**.mpe" -X "[PROJECT-PATH]/tungag/**.mpg" -X "[PROJECT-PATH]/tungag/**.mpeg" -X "[PROJECT-PATH]/tungag/**.mpv2" -X "[PROJECT-PATH]/tungag/**.mp2" -X "[PROJECT-PATH]/tungag/**.mp4" -X "[PROJECT-PATH]/tungag/**.mov" -X "[PROJECT-PATH]/tungag/**.wmv" -X "[PROJECT-PATH]/tungag/**.rm" -X "[PROJECT-PATH]/tungag/**.avi" -X "[PROJECT-PATH]/tungag/**.wvx" -X "[PROJECT-PATH]/tungag/**.m1v"
Executing: add  -I "[PROJECT-PATH]/tungag/**.ChorusNotes" -I "[PROJECT-PATH]/tungag/FLExProject.ModelVersion" -I "[PROJECT-PATH]/tungag/FLExProject.CustomProperties" -I "[PROJECT-PATH]/tungag/do_not_share_project.txt" -I "[PROJECT-PATH]/tungag/.hgignore" -I "[PROJECT-PATH]/tungag/**.list" -I "[PROJECT-PATH]/tungag/**.style" -I "[PROJECT-PATH]/tungag/LinkedFiles/AudioVisual/**.*" -I "[PROJECT-PATH]/tungag/LinkedFiles/Others/**.*" -I "[PROJECT-PATH]/tungag/LinkedFiles/Pictures/**.*" -I "[PROJECT-PATH]/tungag/SupportingFiles/**.*" -I "[PROJECT-PATH]/tungag/CachedSettings/ConfigurationSettings/*.fwlayout" -I "[PROJECT-PATH]/tungag/CachedSettings/ConfigurationSettings/**.fwdictconfig" -I "[PROJECT-PATH]/tungag/CachedSettings/ConfigurationSettings/**Overrides.css" -I "[PROJECT-PATH]/tungag/CachedSettings/WritingSystemStore/*.ldml" -I "[PROJECT-PATH]/tungag/CachedSettings/SharedSettings/*.plsx" -I "[PROJECT-PATH]/tungag/Linguistics/Reversals/**.reversal" -I "[PROJECT-PATH]/tungag/Linguistics/Lexicon/*.lexdb" -I "[PROJECT-PATH]/tungag/Linguistics/TextCorpus/*.textincorpus" -I "[PROJECT-PATH]/tungag/Linguistics/Inventory/*.inventory" -I "[PROJECT-PATH]/tungag/Linguistics/Discourse/Charting.discourse" -I "[PROJECT-PATH]/tungag/Linguistics/**.featsys" -I "[PROJECT-PATH]/tungag/Linguistics/MorphologyAndSyntax/AnalyzingAgents.agents" -I "[PROJECT-PATH]/tungag/Linguistics/MorphologyAndSyntax/MorphAndSynData.morphdata" -I "[PROJECT-PATH]/tungag/Linguistics/Phonology/PhonologicalData.phondata" -I "[PROJECT-PATH]/tungag/Anthropology/DataNotebook.ntbk" -I "[PROJECT-PATH]/tungag/Other/ReferenceSystem.srs" -I "[PROJECT-PATH]/tungag/Other/*.ArchivedDraft" -I "[PROJECT-PATH]/tungag/Other/Translations.trans" -I "[PROJECT-PATH]/tungag/Other/Settings.ImportSetting" -I "[PROJECT-PATH]/tungag/Other/Books/*.bookannotations" -I "[PROJECT-PATH]/tungag/Other/Books/*.book" -I "[PROJECT-PATH]/tungag/General/FLExFilters.filter" -I "[PROJECT-PATH]/tungag/General/FLExAnnotations.annotation" -I "[PROJECT-PATH]/tungag/General/LanguageProject.langproj" -I "[PROJECT-PATH]/tungag/General/UnownedPictures.pictures" -I "[PROJECT-PATH]/tungag/General/VirtualOrdering.orderings" -I "[PROJECT-PATH]/tungag/**.ChorusRescuedFile" -X "[PROJECT-PATH]/tungag/~~*.txt" -X "[PROJECT-PATH]/tungag/**.NewChorusNotes" -X "[PROJECT-PATH]/tungag/**.fwdata" -X "[PROJECT-PATH]/tungag/**.fwdata-replaced" -X "[PROJECT-PATH]/tungag/**.fwdata-x" -X "[PROJECT-PATH]/tungag/**.fwdb" -X "[PROJECT-PATH]/tungag/**.bad" -X "[PROJECT-PATH]/tungag/**.bak" -X "[PROJECT-PATH]/tungag/**.flextext" -X "[PROJECT-PATH]/tungag/**.fwbackup" -X "[PROJECT-PATH]/tungag/**.fwstub" -X "[PROJECT-PATH]/tungag/**.lint" -X "[PROJECT-PATH]/tungag/**.log" -X "[PROJECT-PATH]/tungag/**.orig" -X "[PROJECT-PATH]/tungag/**.oxekt" -X "[PROJECT-PATH]/tungag/**.oxes" -X "[PROJECT-PATH]/tungag/**.oxesa" -X "[PROJECT-PATH]/tungag/**.tmp" -X "[PROJECT-PATH]/tungag/**.xml" -X "[PROJECT-PATH]/tungag/**.zip" -X "[PROJECT-PATH]/tungag/**.fwdata.lock" -X "[PROJECT-PATH]/tungag/**.dupid" -X "[PROJECT-PATH]/tungag/Temp/**.*" -X "[PROJECT-PATH]/tungag/BackupSettings/**.*" -X "[PROJECT-PATH]/tungag/WritingSystemStore/trash/**.*" -X "[PROJECT-PATH]/tungag/WritingSystemStore/WritingSystemsToIgnore.xml.ChorusNotes" -X "[PROJECT-PATH]/tungag/OtherRepositories/**.*" -X "[PROJECT-PATH]/tungag/**.mpa" -X "[PROJECT-PATH]/tungag/**.mpe" -X "[PROJECT-PATH]/tungag/**.mpg" -X "[PROJECT-PATH]/tungag/**.mpeg" -X "[PROJECT-PATH]/tungag/**.mpv2" -X "[PROJECT-PATH]/tungag/**.mp2" -X "[PROJECT-PATH]/tungag/**.mp4" -X "[PROJECT-PATH]/tungag/**.mov" -X "[PROJECT-PATH]/tungag/**.wmv" -X "[PROJECT-PATH]/tungag/**.rm" -X "[PROJECT-PATH]/tungag/**.avi" -X "[PROJECT-PATH]/tungag/**.wvx" -X "[PROJECT-PATH]/tungag/**.m1v" 
Committing "[FLExBridge: 1.0.0] sync"
mark committing with comment: [FLExBridge: 1.0.0] sync
Executing: ci -u "mark" -m "[FLExBridge: 1.0.0] sync" 

Executing: status --change tip
standout: M CachedSettings/WritingSystemStore/lcm.ldml
M General/LanguageProject.langproj
M Linguistics/Lexicon/Lexicon_03.lexdb
M WritingSystemStore/lcm.ldml

M CachedSettings/WritingSystemStore/lcm.ldml
M General/LanguageProject.langproj
M Linguistics/Lexicon/Lexicon_03.lexdb
M WritingSystemStore/lcm.ldml
Validating CachedSettings/WritingSystemStore/lcm.ldml
Validating General/LanguageProject.langproj
Validating Linguistics/Lexicon/Lexicon_03.lexdb
Validating WritingSystemStore/lcm.ldml
Executing: parents --template "changeset:{rev}:{node|short}
branch:{branches}
user:{author}
date:{date|rfc822date}
tag:{tags}
summary:{desc}
parent:{p1rev}:{p1node}"
standout: changeset:127:68a92d84c743
branch:7500002.7000072
user:mark
date:Tue, 30 Jan 2024 16:47:04 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync
parent:126:dc2ebed055e6a4760ead4036486d890eeda7dc74

changeset:127:68a92d84c743
branch:7500002.7000072
user:mark
date:Tue, 30 Jan 2024 16:47:04 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync
parent:126:dc2ebed055e6a4760ead4036486d890eeda7dc74
Executing and caching: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Executing and caching: log -r127 --template "{node}" 
standout: 68a92d84c743ff4c99f6f12d12ebddffa9e9ca34

Using cached result: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Executing and caching: log -r126 --template "{node}" 
standout: dc2ebed055e6a4760ead4036486d890eeda7dc74

Using cached result: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Connecting to hg.archraspberrypi.tk...
Using cached result: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Pinging hg.archraspberrypi.tk...
Ping took 36 milliseconds
Receiving any changes from hg.archraspberrypi.tk
(hg.archraspberrypi.tk is http://flex:********@hg.archraspberrypi.tk/fieldwork_language_explorer)
Initiating Normal Transport
Executing: tip --template "changeset:{rev}:{node|short}
branch:{branches}
user:{author}
date:{date|rfc822date}
tag:{tags}
summary:{desc}
"
standout: changeset:127:68a92d84c743
branch:7500002.7000072
user:mark
date:Tue, 30 Jan 2024 16:47:04 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync

changeset:127:68a92d84c743
branch:7500002.7000072
user:mark
date:Tue, 30 Jan 2024 16:47:04 +0100
tag:tip
summary:[FLExBridge: 1.0.0] sync
Using cached result: log -r0 --template "{node}" 
standout: a62f6a643784668b70be7f081c1206ed5fe3db88

Using cached result: log -r127 --template "{node}" 
standout: 68a92d84c743ff4c99f6f12d12ebddffa9e9ca34

Checking for proxy by trying to http-get http://proxycheck.palaso.org...
RobustNetworkOperation.Do()
Executing: pull "http://flex:********@hg.archraspberrypi.tk/fieldwork_language_explorer" 
standerr: abort: error: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:727)

Executing: version
standout: Mercurial Distributed SCM (version 3.0.1)
(see http://mercurial.selenic.com for more information)

Copyright (C) 2005-2014 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Mercurial Distributed SCM (version 3.0.1)
(see http://mercurial.selenic.com for more information)

Copyright (C) 2005-2014 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Warning: Could not receive from hg.archraspberrypi.tk
Exception: abort: error: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:727)

hg command was
pull "http://flex:********@hg.archraspberrypi.tk/fieldwork_language_explorer" 
hg version is Mercurial Distributed SCM (version 3.0.1)
(see http://mercurial.selenic.com for more information)


  at Chorus.VcsDrivers.Mercurial.HgRepository.PullFromTarget (System.String targetLabel, System.String targetUri) [0x00112] in <03c3e6c8d3114a508600fb24b8226210>:0 
  at Chorus.VcsDrivers.Mercurial.HgNormalTransport.Pull () [0x00000] in <03c3e6c8d3114a508600fb24b8226210>:0 
  at Chorus.VcsDrivers.Mercurial.HgRepository.Pull (Chorus.VcsDrivers.RepositoryAddress source, System.String targetUri) [0x00055] in <03c3e6c8d3114a508600fb24b8226210>:0 
  at Chorus.sync.Synchronizer.PullFromOneSource (Chorus.VcsDrivers.Mercurial.HgRepository repo, Chorus.VcsDrivers.RepositoryAddress source, System.Collections.Generic.Dictionary`2[TKey,TValue] connectionAttempt) [0x00102] in <03c3e6c8d3114a508600fb24b8226210>:0 
  at Chorus.sync.Synchronizer.PullFromOthers (Chorus.VcsDrivers.Mercurial.HgRepository repo, System.Collections.Generic.List`1[T] sourcesToTry, System.Collections.Generic.Dictionary`2[TKey,TValue] connectionAttempt) [0x0001e] in <03c3e6c8d3114a508600fb24b8226210>:0 
  at Chorus.sync.Synchronizer.SyncNow (Chorus.sync.SyncOptions options) [0x00081] in <03c3e6c8d3114a508600fb24b8226210>:0 
abort: error: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:727)

hg command was
pull "http://flex:********@hg.archraspberrypi.tk/fieldwork_language_explorer" 
hg version is Mercurial Distributed SCM (version 3.0.1)
(see http://mercurial.selenic.com for more information)

Trivially, if I disable the redirect on my server from the http:// to https:// URL, then sync works fine. Also, if I allow deprecated TLSv1, it also works. However, I would rather avoid doing either of those, as they seem like bad solutions to what should be fixable on mercurial's end.

The mercurial version that is bundled in the Flatpak is obviously very very old... 3.0.1 dates back to June 2014, and we are now on 6.5.1! I understand there had probably been a lot of development going on both in Chorus and Fieldworks since then, and that this issue is likely not present in more recent version, but since they aren't available to the Flatpak installers such as myself, is there a way anybody can think of of working around this? I'm open to tinkering and testing stuff. Even rebuilding a flatpak or whatever.

Any help or suggestions would be much appreciated.
Best regards,
Mark.

Encoding issues displaying conflict reports on Windows

When Chorus generates HTML for a conflict report, it (currently) does not declare an encoding. This results in GeckoFX on Windows assuming an encoding of ISO-8859-1 or CP1252 for the displayed HTML, which results in incorrect rendering of non-Roman text:

If a <meta> tag is added to Chorus's generated HTML explicitly specifying UTF-8, then the text renders correctly:

The GetHtml() method in Chorus.FileTypeHandlers.ConflictPresenter should add an HTML <meta> tag specifying UTF-8.

exception handling around GetDrives()?

Some of the users of Chorus in OSE get the below exception stack (green screen of death). I think in the past it was when maybe they had something plugged into a USB port... Is it possible to add exception catching if GetDrives() fails?

Msg: Invalid class 
Class: System.Management.ManagementException
Source: System.Management
Assembly: System.Management, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Stack:    at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)
   at System.Management.ManagementObjectCollection.ManagementObjectEnumerator.MoveNext()
   at SIL.UsbDrive.Windows.UsbDriveInfoWindows.GetDrives() in C:\BuildAgent\work\a868e9f1d5fee9d4\SIL.Core.Desktop\UsbDrive\Windows\UsbDriveInfoWindows.cs:line 69
   at SIL.UsbDrive.UsbDriveInfo.GetDrives() in C:\BuildAgent\work\a868e9f1d5fee9d4\SIL.Core.Desktop\UsbDrive\UsbDriveInfo.cs:line 112
   at Chorus.UI.UsbDriveLocator.ScanForUsbDrives()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
Thread: 
Thread UI culture: en-US
Exception: System.Management.ManagementException

--Error Reporting Properties--
Version: Version 4.2.7 (apparent build date: 30-Mar-2021)
CommandLine: "C:\Program Files (x86)\SIL\OneStory Editor\StoryEditor.exe" 
CurrentDirectory: C:\Program Files (x86)\SIL\OneStory Editor
MachineName: ARIKA
OSVersion: Windows 10 
DotNetVersion: 4.0.30319.42000
WorkingSet: 25501696
UserDomainName: ARIKA
UserName: Admin
Culture: en-US
KeyboardAdaptors: WinKeyboardAdaptor

--Log--
Thursday, September 2, 2021
9:09:45 AM	App Launched with ["C:\Program Files (x86)\SIL\OneStory Editor\StoryEditor.exe" ]
9:09:46 AM	Keyboard adaptors in use: WinKeyboardAdaptor
9:09:47 AM	Exception: Invalid class 
Details of most recent events:
9:09:45 AM	App Launched with ["C:\Program Files (x86)\SIL\OneStory Editor\StoryEditor.exe" ]
9:09:46 AM	Keyboard adaptors in use: WinKeyboardAdaptor
9:09:47 AM	Exception: Invalid class 


To Reproduce:

Send/Receive credentials logged in clear text

The Send/Receive URL(s) logged here log the entire (encoded) URL in cleartext including the <username>:<password>:

chorus/src/LibChorus/VcsDrivers/Mercurial/HgRepository.cs

Line 1061 in 04e1e28

 _progress.WriteVerbose($"Repository URI: {string.Join(Environment.NewLine, potentialAddresses.Select(RepositoryURIForLog))}"); 

e.g.:

Getting project...&#xD;
   Started at 2024-04-12 11:59:27Z&#xD;
   Local User: tim&#xD;
   LanguageForge User: admin&#xD;
   Repository URI: https://admin:[email protected]:443/elawa-dev-flex&#xD;
   Local Directory: C:\Users\tim\AppData\Local\Temp\SR_Tests\CloneBigProject\1-9eff4b07\elawa-dev-flex&#xD;&#xD;

Convert ChorusMerge to .NET6

Blocked waiting on authentication/permissions for nuget packaging via the new GH Actions script

Compile for ARM-based Linux?

Hello there, wondering if chorus hub could be recompiled for Linux running on an ARM chip? I'm looking at outfitting a new remote translation house with a Raspberry pi 3 just to be a low power server for chorus hub and some file sharing. I was bummed to find I couldn't run chorus on an ARM processor. Not sure what all your dependencies look like, but thought I'd ask the question and see if someone would be willing to compile the package for ARM. Thanks.

change languagedepot.org url to lexbox.org

this will change the domain used in the dialog, but we also want to change any pre existing projects. This will require hooking this ini check code

chorus/src/LibChorus/VcsDrivers/Mercurial/HgRepository.cs

Lines 155 to 176 in 104ce40

 internal void CheckAndUpdateHgrc() 

 { 

 CheckMercurialIni(); 

 if (!_hgrcUpdateNeeded) 

 return; 

 try 

 { 

 lock(_pathToRepository) // Avoid crash if two threads try to Sync simultaneously 

 { 

 EnsureChorusMergeAddedToHgrc(); 

 EnsureCacertsIsSet(); 

 var extensions = HgExtensions; 

 EnsureTheseExtensionsAndFormatSet(extensions); 

 _hgrcUpdateNeeded = false; 

 } 

 } 

 catch (Exception error) 

 { 

 throw new ApplicationException($"Failed to set up extensions for the repository: {error.Message}", error); 

 } 

 }

if the domain is Ld then change it to lexbox. Be careful to make sure it's not changing the domain in other cases.

I'd also like to try and change how the protocol logic works so we can drop all the domains except lexbox. This will require changing

chorus/src/LibChorus/VcsDrivers/RepositoryAddress.cs

Lines 108 to 114 in 7504501

 public static bool IsKnownResumableRepository(string uri) 

 { 

 // REVIEW (Hasso) 2021.01: this seems to apply only to HTTP repositories, and is now stored in a separate property of 

 // ServerSettingsModel.cs. Perhaps we should move IsResumable to HttpRepositoryAddress and this logic to determine 

 // resumability to ServerSettingsModel. 

 return uri.ToLower().Contains("resumable"); 

 }

and any code that calls it to use a stored value of what protocol to use for a project (or it might be global?).

For HG It will also require changing the assumption that the url is https:://{host}/{projectId} to https:://{host}/hg/{projectId} (project id being the chorus term, lexbox calls it project code)

chorus/src/LibChorus/Model/ServerSettingsModel.cs

Line 127 in 80bee2f

ProjectId = WebUtility.UrlDecode(UrlHelper.GetPathAfterHost(url));

and

chorus/src/LibChorus/Model/ServerSettingsModel.cs

Line 149 in 80bee2f

return $"https://{Host}/{WebUtility.UrlEncode(ProjectId)}";

though there may be other places.

Chorus fails to resolve merge conflict when syncing in Language Forge

log.txt

Above is the log output from LfMerge. It's referencing SIL.ChorusPlugin.LfMergeBridge v3.8.0-beta*. It also references SIL.Chorus.ChorusMerge v5.0.0-beta0030, but I'm not exactly sure how that reference is being used. Here's the exception captured from Chorus:

2022-09-02T06:17:24.4077801Z Error:Sync failure: Chorus.sync.Synchronizer+SynchronizationException: Unable to complete the send/receive.
 ---> System.ApplicationException: warning: conflicts during merge.
merging General/LanguageProject.langproj incomplete! (edit conflicts, then use 'hg resolve --mark')
warning: conflicts during merge.
merging Linguistics/Lexicon/Lexicon_06.lexdb incomplete! (edit conflicts, then use 'hg resolve --mark')

   at Chorus.VcsDrivers.Mercurial.HgRepository.Merge(String localRepositoryPath, String revisionNumber)
   at Chorus.sync.Synchronizer.MergeTwoChangeSets(Revision head, Revision theirHead)
   at Chorus.sync.Synchronizer.MergeHeads()
   --- End of inner exception stack trace ---
   at Chorus.sync.Synchronizer.ExplainAndThrow(Exception exception, WhatToDo whatToDo, String explanation, Object[] args)
   at Chorus.sync.Synchronizer.MergeHeads()
   at Chorus.sync.Synchronizer.MergeHeadsOrRollbackAndThrow(HgRepository repo, Revision workingRevBeforeSync)
   at Chorus.sync.Synchronizer.SyncNow(SyncOptions options)

My test case is as follows:

Add entry in LF and sync to Language Depot
Sync and verify change in FieldWorks
Modify a text field of that entry in LF
Modify the same field in LD and sync
Sync in LF

Build ChorusMergeModule (and ChorusHubInstaller?)

When we moved the build to GitHub Actions building the ChorusMergeModule and ChorusHubInstaller was accidentally left out.

We should add building the ChorusMergeModule to dotnet.yml and save it as an artifact.

Also check if the ChorusHubInstaller is still needed - in a discussion on slack it was noted:

Looks like ChorusHub was last released for Windows in 2018 and the most recent successful TC build was in 2017. I don't know what the current process is, if any.
getting rid of build/ may be safe. After so much bit rot, it may be easier to rewrite the installer from scratch if we need to.

However, it seems that ChorusMergeModule.msm and ChorusHubInstaller.msi got build as part of the regular builds (targets Installer ChorusHubInstaller).

Check build/Chorus.proj prior to commit e7ab5bc and TC for how it was done in the past.

ToDo

Add building ChorusMergeModule to dotnet.yml
Is ChorusHubInstaller still needed/used by anyone?
If it is, add building ChorusHubInstaller to dotnet.yml
document (e.g. in dotnet.yml) where ChorusMergeModule (and ChorusHubInstaller) is used
(in FieldWorks and FLEx Bridge installers)
(nice to have) automatically stamp copyright dates on both
Remove Build/Chorus.proj

Chorus Hub should not fail if multiple instances are running

This is a report on behalf of Peter Wulfing...
Currently, if multiple instances of Chorus Hub are running, it does not work. Also, it is difficult to determine that more than one instance is running without examining each person's computer.

Possible fixes:

Chorus Hub should notify the user that multiple instances are running and allow him/her to choose which one they want to use.
John Hatton proposed a less-expensive fix: All Chorus Hub instances should detect each other and put up a message saying that you have to turn off all but one.

Included Mercurial doesn't work on Debian Wheezy

Running the included Mercurial on Debian Wheezy fails because Wheezy uses a different version of glibc.

Password encoding is wrong in ServerSettingsModel.LogIn

The LogIn() method in ServerSettingsModel currently contains the following code:

chorus/src/LibChorus/Model/ServerSettingsModel.cs

Lines 338 to 342 in 271b640

 var passwordBytes = Encoding.UTF8.GetBytes($"password={Password}"); 

 request.ContentType = "application/x-www-form-urlencoded"; 

 request.ContentLength = passwordBytes.Length; 

 var passwordStream = request.GetRequestStream(); 

 passwordStream.Write(passwordBytes, 0, passwordBytes.Length);

Notice how it's declaring the content type as "application/x-www-form-urlencoded" but not actually URL-encoding the password. This violates the spec which says:

Forms submitted with this content type must be encoded as follows:
Control names and values are escaped. Space characters are replaced by +, and then reserved characters are escaped as described in RFC1738, section 2.2: Non-alphanumeric characters are replaced by %HH, a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., %0D%0A).

This is the root cause of issues such as sillsdev/languageforge-lexbox#255, because it is easy to generate passwords with a password generator that contain characters such as & and + (that's not an exhaustive list), which will end up being sent incorrectly by Chorus and then rejected with a password error on the other end (when "a+b" gets URL-decoded into "a b", for example).

The long-term solution to this bug is to fix how Chorus sends passwords via HTTP POST, by URL-encoding them before writing them into the POST request body. The short-term solution will have to be telling users not to put special characters into their passwords. Some special characters make it through unscathed, due to not having any special meaning when they are URL-decoded, but it's far easier to explain "no punctuation in your password" in a choose-your-password UI than to explain "the & character, and these other characters, will fail because we messed up our code".

Packaging problem on Chorus Hub for Linux

This is a report on behalf of Peter Wulfing...

ChorusHub has a packaging problem on Linux:

There was an error in the script that Peter fixed. (There is an erroneous space between two spaces in the script where it was indicating a parent directory in the path.)
Also, there is a missing but required library package (mono-4.0-service) that he had to find and install himself.

Extract VCSDriver independent interface

In the long run it would be nice to be able to choose whether to use Mercurial or Git as VCS backend.

As a first step we should refactor things so that we have a VCSDriver independent interface with a method to select which driver to use. Currently we have e.g. HgRepository class which has several static methods.

GHA workflow for l10n failing due to missing secret

The workflow added in #326 has been failing to run because secrets.CROWDIN_PAT is empty. See this build log where the crowdin download --all -T ${{ secrets.CROWDIN_PAT }} step is turning into crowdin download sources -T due to the missing CROWDIN_PAR secret, which then results in the crowdin tool complaining about a missing required parameter for the -T option.

HgRunner/MercurialLocations point to wrong executable on Windows

Using HgRunner.Run() on Windows I get:

[test:integration]    System.ComponentModel.Win32Exception : An error occurred trying to start process 'D:\code\languageforge-lexbox\backend\Testing\bin\Debug\net8.0\Mercurial\hg' with working directory 'D:\code\languageforge-lexbox\backend\Testing\bin\Debug\net8.0'. The specified executable is not a valid application for this OS platform.
[test:integration]   Stack Trace:
[test:integration]      at Chorus.VcsDrivers.Mercurial.HgRunner.Run(String commandLine, String fromDirectory, Int32 secondsBeforeTimeOut, IProgress progress) in D:\code\chorus\src\LibChorus\VcsDrivers\Mercurial\HgRunner.cs:line 95

Because it always explicitly points to hg instead of hg.exe

Tweak GitHub Actions workflow to run tests on multiple TargetFrameworks in parallel

Currently the Windows tests are run against net461 and net6.0, and #332 adds net8.0. Once #332 is merged it will take about 2 hours to run the Windows tests, which run in series rather than in parallel. It would be nice to use a matrix to specify the target framework to run the tests against, so that net461, net6.0, and net8.0 tests can all run in parallel on different runners. This would take the runtime of the CI build from 2 hours to roughly 40-45 minutes.

Workflow using very old versions of upload-artifact, will stop working June 30, 2024

See https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ — we need to update to v4 of the upload-artifact action or our NuGet workflow will stop working after 2024-06-30.

	if (_outputReader != null)
	_outputReader.Abort();
	if (_errorReader != null)
	_errorReader.Abort();

	internal void CheckAndUpdateHgrc()
	{
	CheckMercurialIni();
	if (!_hgrcUpdateNeeded)
	return;

	try
	{
	lock(_pathToRepository) // Avoid crash if two threads try to Sync simultaneously
	{
	EnsureChorusMergeAddedToHgrc();
	EnsureCacertsIsSet();
	var extensions = HgExtensions;
	EnsureTheseExtensionsAndFormatSet(extensions);
	_hgrcUpdateNeeded = false;
	}
	}
	catch (Exception error)
	{
	throw new ApplicationException($"Failed to set up extensions for the repository: {error.Message}", error);
	}
	}

	public static bool IsKnownResumableRepository(string uri)
	{
	// REVIEW (Hasso) 2021.01: this seems to apply only to HTTP repositories, and is now stored in a separate property of
	// ServerSettingsModel.cs. Perhaps we should move IsResumable to HttpRepositoryAddress and this logic to determine
	// resumability to ServerSettingsModel.
	return uri.ToLower().Contains("resumable");
	}

	var passwordBytes = Encoding.UTF8.GetBytes($"password={Password}");
	request.ContentType = "application/x-www-form-urlencoded";
	request.ContentLength = passwordBytes.Length;
	var passwordStream = request.GetRequestStream();
	passwordStream.Write(passwordBytes, 0, passwordBytes.Length);

sillsdev / chorus Goto Github PK

chorus's People

Contributors

Stargazers

Watchers

Forkers

chorus's Issues

ToDo

Recommend Projects

Recommend Topics

Recommend Org