Opaque failure - What to do when 'Learning fails'? about prose HOT 4 CLOSED

Since a program could not be learned from the examples given, usually, more examples will not help. Since normally all programs expressible in the DSL which satisfy the examples are learned, no programs learned means that there are no programs in the DSL that satisfy all of the examples and adding more examples would only further constraint the learning problem. (I say "normally" because using the escape hatches of the learning procedure you could write your own non-monotonic learning sub-procedure... but that's generally a bad idea because of the confusion you bring up.) As you say, this means that the grammar would likely have to be extended to express the desired operation.

We know the error reporting is poor and it's an issue we intend to address.

If you are comfortable sharing your data, it would be helpful to see your inputs, both to determine if it is in fact not expressible and, if so, help us know how we might want to extend the language to cover your scenario. You can e-mail me at [email protected] if you don't want to share it publicly.

from prose.

@danpere Thanks for the feedback. I am not sure, but I think one of the problems I might be running into is that there are (at least) two date formats mingled in the documents yyyymmdd and mm/dd/yyyy, either of which could be the accepted 'output' and the grammar may be failing to generalize across them.

from prose.

For clarification, you are using the Extraction.Text language? (That's the sample that has that exact text as the error message.)

The differing formats might be the issue. Extraction.Text usually ends up being able to use context when the formats are different, but that might not apply in this case. Also, there is a regular expression internally for matching "dates" which is fairly flexible, but it can't cover everything. Extraction.Text does not currently support conditionals, but one way to work around that is to make multiple fields for the different date formats/contexts. @vuminhle may be able to give more tips on getting it to work on difficult scenarios.

from prose.

@danpere has covered all the main points.
If there is no program, it means that your task cannot be expressed in the current grammar. We could have given you back the problematic examples (or a maximal subset of working examples), but I'm not sure if that information is useful. Furthermore, there may be more than one variations of such sets.
We do give more indicative messages if your examples are conflicting or duplicating.

As you rightly observed, we can solve this by extending the grammar to support the task. @danpere mentioned learning conditional, which basically partitions your inputs into different clusters (each of which shares the same format) and learns a program for each of them. This is on-going work.

Which API did you use? Did you extract a substring out of a string, or a sequence of substrings out of a string?
It would be great if you can share one or two lines of your (anonymized) data, together with the fields you are extracting, so that we can analyze what is going on.

from prose.