kimi-arthur / kifanet Goto Github PK

Currently we retrieve all words from Memrise once per invocation of memx, which takes quite long, especially for insertion of only a few words. This was to make sure we have latest correct data. However, that may not be necessary as we check for specific words. The risk would come if one word is deleted without us knowing. We can try remedy that later.

One question is whether we should use ThingId or self constructed course_name/word. Since we may need to provide for other setups, the first choice may be better. But we should decide on that before proceeding.

Current ones:

• configs/baidu_cloud
• configs/mega_nz
• jobs

Also evaluate whether they are still needed or not.

Since we have api likes FindExistingFiles, the legacy framework like approach of KifaFileCommand should be deprecated.

Potentially a solution for dotnet/runtime#3858, which may only work for non-null properties, based on source generators.

It's still quite useful, given that most of the NullReferenceExceptions come from the following sources (with the exception of class's fields)

1. Local variable is unexpectedly null
2. Return value of a method is unexpectedly null
3. Property retrieved (not called) is unexpectedly null

While new unexpected nulls come from 3 if all obvious warnings are fixed (e.g. returning nullable value in a method). While you can always initialize a non-null property to a value, it is not always reasonable to do so (e.g. inializing a string property to "" while it actually has to be specified by client code).
One suggestion from Microsoft is to initialize the property to null!. However, in this case, checker and compiler won't do anything, and runtime won't do anything either, even when the null value is obtained. Instead it only produces useless NullReferenceException: Object reference not set to an instance of an object error when its members are used.

The option is to declare the property's getter as => _backField ?? throw new InvalidOperationException("Uninitialized property: " + nameof(ShippingAddress)), which is much nicer, but needs quite some boilerplate than the option above.

This issue will try to use Source Generator to provide a clean solution for any property.

For any non-null property,

public string Id { get; set; }

It will produce the property as

private string? _id;

public string Id {
  set {
    if (value == null) {
        throw new NullReferenceException("Cannot assign null value to non-null property Id of class xxx");
    }
    _id = value;
  }

  get => _id ?? throw new NullReferenceException("Property Id of class xxx is expected to be non-null, but is actually null.");
}

This error message is already much more helpful than nothing (the current state). Normally there won't two accesses of the same property of different objects in the same line. Two caveats exist though,

1. It is not actually a NullReferenceException, but rather a precaution beforehand (maybe InvalidOperationException is more reasonable?). The code may not fail for NullReferenceException if without the added check. However, when the exception is thrown here, it's definitely a codesmell and should be fixed.
2. It cannot pinpoint and notify which variable is null in the first place (like Java 14's solution).

So with this feature, it works something like lateinit in Kotlin or late in Dart.

Potential improvements include

1. Make the property required when defining the class

Generally, our data is from some data source. It's supposed to be able to refresh and get the same result back (controlled via Fill method).

However, we may need to override the data if we cannot directly modify the source. For example, we may not want the way a TV episode's name is rendered. In this case, we have two options, one is for general conversion that we add the logic to run every time for every item. The other one would be for individual items that we record how we want to modify the data and apply it whenever it's reFilled.

Say we have a data object like

{
  "id": "Merlin",
  "overview": "An epic story about a warlock.",
  "seasons": [
    {
      "id": "1",
      "title": "First Season"
    }
  ]
}

And we want to override the title to Season one. For that, we should be able to call an override API like, PUT /tv_shows/<id>, with a Dictionary<string, object>:

{
  "overview": "new overview",
  "seasons/0/title": "new title"
}

The stored data should look like,

{
  "$metadata": {
    "overview": "new overview",
    "seasons/0/title": "new title"
  },
  "id": "Merlin",
  "overview": "new overview",
  "seasons": [
    {
      "id": "1",
      "title": "new title"
    }
  ]
}

And no matter how the data changes, the specified fields will be overridden afterwards.

This can potentially be merged with PATCH, but no details are thought through.

Currently we do this manually via declaration and runtime overrides and with mutiple flavors.

For case 1, we can do something like here and here. We may even skip the client like here. We also create the clients by ourselves like here, which made it very messy.

For case 2, we have here and here.

The reason for different structures of clients was because we had a Python based server, that fills the gap of functionalities. Since we don't have that now. It seems to reasonable to just set which set of clients to use in each scenario. Namely, if it's server, it's all JsonClient, otherwise all RestClient. However, how to set that up cleanly is still to be investigated. It would be great if we can just link the type to its client automacally.

Currently, it's tedious to call a simple method:

        var arguments =
            $"-safe 0 -f concat -i \"{fileListFile.GetLocalPath()}\" -i \"{cover.GetLocalPath()}\" -map 1 -disposition:v:0 attached_pic -map 0 -c copy \"{target.GetLocalPath()}\"";
        Logger.Trace($"Executing: ffmpeg {arguments}");
        using var proc = new Process {
            StartInfo = {
                FileName = "ffmpeg",
                Arguments = arguments
            }
        };
        proc.Start();
        proc.WaitForExit();
        if (proc.ExitCode != 0) {
            throw new Exception("Merging files failed.");
        }

We do copy paste and the messages are not well handled (should be redirected etc.). Better if we have a centralized control and we can just call something like

Kifa.Execute("ffmpeg", $"-safe 0 -f concat -i \"{fileListFile.GetLocalPath()}\" -i \"{cover.GetLocalPath()}\" -map 1 -disposition:v:0 attached_pic -map 0 -c copy \"{target.GetLocalPath()}\"");

And the rest is properly handled.