Here's my situation: I'm writing something that takes natural commands, like chalkbot, set my timezone to America/Los_Angeles/チョークボット、タイムゾーンをアメリカのロサンゼルスに設定して or chalkbot remind 24h1m do stuff. I'm developing on a Windows machine, but I want to move this over to a Linux machine for the "production environment."

I'm worried about

1. The names for the different grammars being such like Eng instead of en_US or equivalent: I'm making use of ICU4J for locales and timezones.
2. JNI, moving my application from Windows to a Linux machine, and maybe wanting to use Graal Native-Image.
3. How to maintain grammars I'll need alongside the application, for example parsing human-readable durations/dates in the remind command.

What should I do/how should I set this up? Perhaps add a backend for directly generating Java code from the grammar files? If I've done a lot of Scala and a little bit of Haskell, would that be easy? Or should I just go the JNI route, compile the Java interface myself, and give up on native-image?

In the split from GF, the release tags seem to have been lost.
They could be re-added manually, although I'm not sure it makes sense to do this. Maybe releases from the old monolithic repository shouldn't be confused with releases post-split.

Conversely, gf-rgl has retained its release tags for 3.8 and 3.9, but maybe these should be removed.

There are missing dependencies in Ubuntu (64-bit) for WSL. See attached file for details
libtinfo5.txt

and some of the packages are also not supported on ubuntu 20. See https://packages.ubuntu.com/search?keywords=libffi6

GF's test suites, invoked with cabal test, should be updated to work again.

GF is on Hackage, which is great. But sometimes you don't need the full power of GF, but it's ok with the Haskell-only PGF runtime (subdir src/runtime/haskell). It even has a cabal file already (pgf.cabal), so I guess it should be fairly easy.

(To be clear: I don't want to remove GF from Hackage, but add another package "PGF")

It feels like the time is becoming [over?]ripe for a new GF release. There's a number of issues/suggestions that I think should be discussed, which is the purpose of this issue.

Version number and milestones

I guess 3.11 should be the next version, that's probably uncontroversial.
We need to decide what milestones we want included in the release. I can think of at least 2 which @inariksit has mentioned elsewhere:

1. Fix all issues related to MonadFail update
2. Get the testsuite working again

Also, the GF binaries include pre-compiled RGL, so we need to decide if there are any upcoming RGL milestones which should block the release.

Release recipe

I think we should document the exact steps to make a release in some appropriately named file like RELEASE.md.
Previously @Thomas-H has taken care of releases, but since he is no longer actively working with GF it is even more important that all information is recorded somewhere.

Some things which this needs to include:

1. Bumping version numbers (and temporarily removing the -git suffix in gf.cabal)
2. Updates to web page (various places)
3. Uploading to Hackage

Hosting binaries on GitHub

Until now, the binaries have been built locally by Thomas and then uploaded to www.grammaticalframework.org. I believe that for the next release, we should have everything built on GitHub and host the binaries there too. Of course we can copy them to the GF server, in case we one day do not use GitHub anymore, but the recent GF server outage has shown that we need to have them stored elsewhere.

I have written GitHub Actions workflows for building Debian and macOS packages (the latter still needs some work). I am hoping it will then be easy to attach those generated artifacts to the releases in GitHub although I don't know too much about this, maybe someone else is willing to help out here.

The problem

Say you are implementing the Foods grammar (any language will do, I chose German arbitrarily) and make the following typo.

concrete FoodsGer of Foods = FoodsI with 
  (Syntax = SyntaxGer),
  (LexFoods = LexFoods) ;

That is, the last line says (LexFoods = LexFoods), instead the expected (LexFoods = LexFoodsGer).

When compiling the grammar, it just hangs without any errors, stopping at the type checking phase.

$ gf -v FoodsGer.gf
…
- parsing FoodsGer.gf
  renaming 
  type checking  

Interestingly, if I typo the line (Syntax=SyntaxGer) as (Syntax=Syntax), I get an error immediately. This gets printed for every function in the grammar, I omitted the rest for brevity.

- parsing FoodsGer.gf
  renaming 
  type checking 

FoodsGer.gf:
   /Users/inari/work/GF/gf-contrib/foods/FoodsI.gf:19:
     Happened in linearization of Wine
      no overload instance of Syntax.mkCN
      with value type {s : Str}
      for argument list
        {s : Number => Case => Str; co : Str; g : Gender; lock_N : {};
         uncap : {s : Number => Case => Str; co : Str}}
      among alternatives
        N
        …
        CN NP
FoodsGer.gf:
   /Users/inari/work/GF/gf-contrib/foods/FoodsI.gf:17:
     Happened in linearization of Very
      cannot infer type of constant very_AdA

Not related to MonadFail

I rolled back to a version from February, and I get the same behaviour. So this cannot be related to the MonadFail fix that caused the infinite loop in #79.

The current version on Hackage does not build with ghc 8.6.5

Reporting this issue from 2016, which hasn't been solved. I get the same error on GF 3.10.4.

The following example based on section 6.9 of the GF book is giving me some trouble.

abstract Binding = {
cat Identifier; Prop;
fun
All : (Identifier -> Prop) -> Prop;
Eq : Identifier -> Identifier -> Prop;
AnIdentifier : Identifier;
}
concrete BindingStr of Binding = {
lincat Prop, Identifier = {s:Str};
lin All b = { s = "(" ++ "All" ++ b.$0 ++ ")" ++ b.s} ;
lin Eq a b = {s = "("++a.s ++ "=" ++ b.s ++ ")" } ;
lin AnIdentifier = {s = "defaultId"};
}

Linearizing works:

linearize (All (\v1 -> Eq v1 v1))

( All v1 ) ( v1 = v1 )

However, parsing fails:

linearize (All (\v1 -> Eq v1 v1)) | parse -cat=Prop
The parsing is successful but the type checking failed with error(s):
Function 'lindef Identifier' is not in scope

Also tried adding "lindef Identifier = \s -> {s = s};", but this still gives the same error.

Also, the first example given in gf-contrib/typetheory/README fails in a similary way.

I am running GF version 3.8. Built on linux/x86_64 with ghc-7.8, flags: interrupt server c-runtime.

Regards,
Bjørnar Luteberget.

Would it be possible to publish the Haskell bindings to the C runtime in Hackage? That would make it easier to distribute GF applications written in Haskell.

Compiling GF the other day with GHC 8.4.4, lts-12.26, I saw some errors which perhaps call for tweaking?

> Building library for gf-3.10..
> [  1 of 163] Compiling CGI
> [  2 of 163] Compiling CGIUtils
> [  3 of 163] Compiling Cache
> [  4 of 163] Compiling Data.Binary.Builder
>
> /private/var/folders/t3/cv27h8kn7wb3xfgpgxdpk7vc0000gn/T/stack95801/gf-3.10/src/runtime/haskell/Data/Binary/Builder.hs:202:40: warning: [-Wdeprecations]
>     In the use of ‘inlinePerformIO’
>     (imported from Data.ByteString.Internal):
>     Deprecated: "If you think you know what you are doing, use 'unsafePerformIO'. If you are sure you know what you are doing, use 'unsafeDupablePerformIO'. If you enjoy sharing an address space with a malevolent agent of chaos, try 'accursedUnutterablePerformIO'."
>     |
> 202 | unsafeLiftIO f =  Builder $ \ k buf -> inlinePerformIO $ do
>     |                                        ^^^^^^^^^^^^^^^
> [  5 of 163] Compiling Data.Binary.Get
>
> /private/var/folders/t3/cv27h8kn7wb3xfgpgxdpk7vc0000gn/T/stack95801/gf-3.10/src/runtime/haskell/Data/Binary/Get.hs:417:14: warning: [-Wdeprecations]
>     In the use of ‘inlinePerformIO’
>     (imported from Data.ByteString.Internal):
>     Deprecated: "If you think you know what you are doing, use 'unsafePerformIO'. If you are sure you know what you are doing, use 'unsafeDupablePerformIO'. If you enjoy sharing an address space with a malevolent agent of chaos, try 'accursedUnutterablePerformIO'."
>     |
> 417 |     return . B.inlinePerformIO $ withForeignPtr fp $ \p -> peek (castPtr $ p `plusPtr` o)
>     |              ^^^^^^^^^^^^^^^^^


> [ 60 of 163] Compiling PGF.TypeCheck
>
> /private/var/folders/t3/cv27h8kn7wb3xfgpgxdpk7vc0000gn/T/stack95801/gf-3.10/src/runtime/haskell/PGF/TypeCheck.hs:44:1: warning: [-Wdeprecations]
>     Module ‘Control.Monad.Error’ is deprecated:
>       Use "Control.Monad.Except" instead
>    |
> 44 | import Control.Monad.Error
>    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

In the reference manual there are links to other locations in the same document which are actually not working. For example in section Names (https://www.grammaticalframework.org/doc/gf-refman.html#names) there is a link (https://www.grammaticalframework.org/doc/gf-refman.html#renaming) which I think is intended to point to section Name resolution but does not work.

Bug.gf:

abstract Bug = {
  cat S ;
  fun
    Test : S ;
    Overtest : S -> S ;
}

BugX.gf:

concrete BugX of Bug = {
  lincat
    S = {s: Str} ;
  lin
    Test = {s = "ok"} ;
    Overtest s = {s = s.s
      ++ table {"test" => "xtest"; _ => []} ! s.s
      } ;
}

Compiling the grammar:

$ gf BugX.gf
                              
         *  *  *              
      *           *           
    *               *         
   *                          
   *                          
   *        * * * * * *       
   *        *         *       
    *       * * * *  *        
      *     *      *          
         *  *  *              
                              
This is GF version 3.10. 
Built on linux/x86_64 with ghc-7.10, flags: interrupt server c-runtime
License: see help -license.   

- compiling Bug.gf...   write file Bug.gfo
- compiling BugX.gf... gf: Internal error in GeneratePMCFG:
    descend (CStr 0,CNil,CProj (LIdent (Id {rawId2utf8 = "s"})) CNil)

Even if the grammar contains errors (seems no), the error should be handled, the compiler shouldn't fail with an internal error.

When #27 and #28 are solved, we can implement grammar transformations. One particular interest is to make a grammar "less strict" (i.e., make it recognise more sentences), by loosening (or removing) some of the parameter dependencies.

When we can print canonical grammars in a machine-readble format (such as YAML, JSON) -- see issue #27 , it should be possible to quickly read the same format into GF.

Since the repo split, gf-core has never succeeded in building on Travis.
It used to work for the old GF repository (sometimes) — someone needs to look into what can be done to get it working again.

@aarneranta @krangelov What is the current semantics of Prelude.nonExist or variants{}? Is it intensional or extensional? (see issue #36 for more details)

• if it is intensional (like the current semantics of variants{...}), then it's only useful at the top-level of a linearisation definition:

    lin f = nonExists;
  

• if it is extensional, then you can use it to state that some inflection forms do not exist:

    lin x = {s = table {Utr => "   "; Neutr => nonExists}}
  

  (in the intensional semantics, this is equivalent to the previous definition)

I think that both nonExist and variants{} use intensional semantics, but please correct me if I'm wrong. In that case, it's a problem because people will think that nonExist is more useful than it really is.

Here are some suggestions on alternative semantics for the two:

1. Let nonExist = variants{}, and use the extensional semantics. This can be implemented by letting nonExist refer to a non-existing token which can never occur (such as &#!$ or something similar).

   • Advantage: both nonExist and variants{} become useful.
   • Disadvantage: variants{} will have different semantics than variants{a;b}.

2. Let nonExist and variants{} denote different semantics (extensional resp. intensional).

   • Advantage: nonExist is useful and variants{} and variants{...} will have the same semantics.
   • Disadvantage: nonExist and variants{} are not interchangeable.

3. Let nonExist = variants{}, and keep using the intensional semantics.

   • Advantage: nonExists and variants{...} will have the same semantics.
   • Disadvantage: not very useful.

Personally I prefer solution (1), but can live with solution (2). I really don't like solution (3).

If we choose the intensional semantics, I propose we only allow the use of variants{} at the top level of a linearisation. I.e., GF should allow this:

lin f = variants{}

but not this:

lin f = {s = variants{}; ...}

The output of gf --help lists the --gfo flag twice, with two distinct meanings:

--gfo, --recomp-if-newer          (default) Recompile from source if the source is newer than the .gfo file.
--gfo, --no-recomp                Never recompile from source, if there is already .gfo file.

I assume this is a typo? Tracked down to this source code:

It would be nice to have proper Haddock documentation for PGF2 available on Hackage.

When I try to build locally (with stack haddock), I get this:

  Missing documentation for:
    showPGF (PGF2.hsc:141)
    ...
    exprHash (PGF2.hs
PGF2/Internal.hsc:532:68: error:
    parse error on input ‘-- ^ Concrete syntax flags’
    |
532 |                                        [(String,Literal)] ->       -- ^ Concrete syntax flags
    |                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
c:355)
    exprSize (PGF2.hsc:362)
    ...

But according to the docs this is the correct way to document function arguments, so I don't get why this is a parse error.

This is VERY niche bug, however encountered two times - one on my laptop (Fedora 30), another on AWS (RedHat 8). I'm guessing it has something to do with RHEL kernel.

Essentialy, it happens when calling PGF_generateAll function. The content of grammar, the language of runtime doesn't make a difference. (Tested on python2.7, python3.6, java)

Tested in both:

• Docker installing .deb
• Compiling gf-core, gf-rgl, runtime from stratch on target machine

Stacktrace from Python:

Program received signal SIGSEGV, Segmentation fault.
0x0000555555992043 in ?? ()
(gdb) backtrace
#0  0x0000555555992043 in ?? ()
#1  0x00007ffff296d4b1 in pgf_generate_all (pgf=<optimized out>, typ=0x555555812ad8, err=err@entry=0x7fffffffbb00, pool=<optimized out>, out_pool=out_pool@entry=0x5555557c15c0) at pgf/reasoner.c:471
#2  0x00007ffff2dbfb7f in PGF_generateAll (self=0x7ffff2feb8d0, args=<optimized out>, keywds=<optimized out>) at pypgf.c:2972
#3  0x00007ffff79ea9b3 in _PyCFunction_FastCallDict () from /lib64/libpython3.6m.so.1.0
#4  0x00007ffff79ead6d in call_function () from /lib64/libpython3.6m.so.1.0
#5  0x00007ffff7a29faa in _PyEval_EvalFrameDefault () from /lib64/libpython3.6m.so.1.0
#6  0x00007ffff79b661a in fast_function () from /lib64/libpython3.6m.so.1.0
#7  0x00007ffff79eae52 in call_function () from /lib64/libpython3.6m.so.1.0
#8  0x00007ffff7a29faa in _PyEval_EvalFrameDefault () from /lib64/libpython3.6m.so.1.0
#9  0x00007ffff797c7db in _PyEval_EvalCodeWithName () from /lib64/libpython3.6m.so.1.0
#10 0x00007ffff79b67d0 in fast_function () from /lib64/libpython3.6m.so.1.0

Consider this lindef using variants:

concrete TestEng of Test = open ParadigmsEng in {
  lin snake_charmer_N = mkN human (variants {mkN "snake-charmer"; mkN "snake charmer"});
}

When I parse it with GF as a library and pretty-print it back, the parenthesis around the variants disappear. Code:

module Test where

import qualified GF
import qualified GF.Support
import qualified Data.Map.Strict as M
import qualified Text.PrettyPrint as PP

main = do
  (name, info) <- GF.getSourceModule GF.Support.noOptions "TestEng.gf"
  let terms = [term | (ident,GF.CncFun _ (Just (GF.L (GF.Local n _) term)) _ _) <- M.toList (GF.jments info) ]
  mapM_ putStrLn $ map (PP.render . GF.ppTerm GF.Terse 0) terms

Output:

mkN human variants {mkN "snake-charmer"; mkN "snake charmer"}

The same thing happens with the | syntax, I guess this is just a bug in printing and not parsing.

Originally reported by Roman Suzi in gf-dev.

When using an overloaded oper with a record as an argument, the order of that record's fields matters:

abstract Test = {
  cat C;
  fun f1, f2, f3, f4, f5: C;
}

concrete TestCnc of Test = {
  param
    Colour = Black | White ;
    Size = Big | Small ;

  oper
    Rec: Type = {
      c: Colour ;
      s: Size ;
    } ;

    mkRec0: Rec -> Rec = \r -> lin C r ;

    mkRec = overload {
      mkRec: Colour -> Size -> Rec = \c,s -> lin C { c = c; s = s } ;
      mkRec: Rec -> Rec = mkRec0 ;
    } ;

  lincat
    C = Rec ;

  lin
    f1 = mkRec White Small ; -- ok

    f2 = mkRec0 { s = Small ; c = White } ; -- ok
    f3 = mkRec0 { c = White ; s = Small } ; -- ok

    f4 = mkRec { s = Small ; c = White } ; -- ok
    f5 = mkRec { c = White ; s = Small } ; -- fails
}

Error for f5:

 TestCnc.gf:28:
   Happened in linearization of f5
    no overload instance of TestCnc.mkRec
    with value type {s : Size; c : Colour}
    for argument list
      {c : Colour; s : Size}
    among alternatives
      Colour Size
      {s : Size; c : Colour}

The enforced order does not seem to be alphabetical (as previously suspected).

The following works fine:

useVV vv =
    let vv2v : VV -> V = \vv' -> lin V vv'
    in lin VP (mkVP (vv2v vv)) ;

but this yields a type error:

 useVV vv = lin VP (mkVP (lin V vv)) ;

Here's the error message:

SecundaRulesLat.gf:
  SecundaRulesLat.gf:18:
    Happened in linearization of useVV
     no overload instance of mkVP
     for
       {act : VActForm => Str; ger : VGerund => Str; geriv : Agr => Str;
        imp : VImpForm => Str; inf : VInfForm => Str; isAux : Bool;
        part : VPartForm => Agr => Str; pass : VPassForm => Str;
        sup : VSupine => Str; lock_V : {}}
     among
       {act : VActForm => Str; ger : VGerund => Str; geriv : Agr => Str;
        imp : VImpForm => Str; inf : VInfForm => Str; lock_V : {};
        part : VPartForm => Agr => Str; pass : VPassForm => Str;
        sup : VSupine => Str}
     ...

The only thing that differs from the types is that the upper has an extra isAux:Bool, which makes it a subtype of the lower. And then it should work, as I understand things, in particular if I use lin V vv as a coercion.

The additional code is:

V = ResLat.Verb ;  -- CatLat.gf
VV = ResLat.VV ;  -- CatLat.gf
VV : Type = Verb ** { isAux : Bool } ;  -- ResLat.gf

When compiling a grammar with --haskell=gadt, it produces a Haskell file with this pragma:

{-# OPTIONS_GHC -fglasgow-exts #-}

With GHC 8.2.2 I get a deprecation warning:

    -fglasgow-exts is deprecated: Use individual extensions instead
  |
1 | {-# OPTIONS_GHC -fglasgow-exts #-}
  |                ^^^^^^^^^^^^^^^^

It should be replaced with:

{-# LANGUAGE FlexibleInstances, KindSignatures, RankNTypes, TypeSynonymInstances #-}

I think it would be great if GitHub could build GF binaries (e.g. as .deb and .pkg) on every commit using GitHub actions. Reasons:

1. Allow users to use latest GF without having to build manually — I myself have wanted this when using GF inside of a Docker container
2. Make releases simpler — building the binaries will no longer be a job that someone has to do
3. Allow us to remove the Travis CI builds and have everything on GitHub

I plan on working on this soon.

There's been discussion about general revamping of error messages, and that's of course useful. Meanwhile, here's a low-hanging fruit: change wording of one of the most baffling ones, the "no overload instance of X for Foo among Foo".

Here's an example of two cases where this error is raised. In the first one, the relevant information really is no overload instance of mkS […] with value type Utt. Everything in the middle, "for Foo among Foo, Foo Bar, …" can be easily removed, and the error message would only get better.

Now, this error is raised as a response to several problems in the code. I don't know if "for Foo among Foo" is actually relevant in some cases, but if it is, at least the error message would improve by changing the order: "no overload instance of mkS with value type Utt for Foo among …".

Maybe some bound is missing on Cabal?
I tried to cabal install with Fedora 31 (ghc-8.6.5) and got:

:
Failed to install gf-3.9
:
cabal: Error: some packages failed to install:
gf-3.9-3SHquerIwkNDVbAmsdivje failed during the configure step. The exception
was:
dieVerbatim: user error (cabal: '/usr/bin/ghc' exited with an error:

/var/home/petersen/tmp/cabal-tmp-227787/gf-3.9/WebSetup.hs:96:28: warning:
[-Wdeprecations]
In the use of ‘die’ (imported from Distribution.Simple.Utils):
Deprecated: "Messages thrown with die can't be controlled with Verbosity; use
die' instead, or dieNoVerbosity if Verbosity truly is not available"
|
96 | die $ command++" exited with exit code: " ++ show i
| ^^^

/var/home/petersen/tmp/cabal-tmp-227787/gf-3.9/dist/setup/setup.hs:378:32:
error:
• Couldn't match type ‘Distribution.System.Platform -> String’
with ‘[Char]’
Expected type: String
Actual type: Distribution.System.Platform -> String
• Probable cause: ‘exeExtension’ is applied to too few arguments
In the second argument of ‘(<.>)’, namely ‘exeExtension’
In the expression: exeName' <.> exeExtension
In an equation for ‘exeNameReal’:
exeNameReal = exeName' <.> exeExtension
|
378 | exeNameReal = exeName' <.> exeExtension
| ^^^^^^^^^^^^
)

The problem

I was poking around Compute.ConcreteNew.hs trying to fix #52, and I noticed that the indices for bound variables (VGen Int [Value]) are not correct. You can even see this in #52: the offending piece of code is this

    x + "foo" => ss (x + "bar")

and the error is this.

unsupported token gluing: str + "bar"

My attempts

I added more printouts for the error message for unsupported token gluing, as follows:

in error . render $
   ppL loc (hang ("unsupported token gluing: "
                 ++ "(" ++ show v1 ++ ", " ++ show v2 ++ ")\n"
                 ++ show (local env) ++ "\n") 4
                  (Glue (vt v1) (vt v2)))

With this change, I can cause an error (unsupported token gluing in my case, both appropriate and inappropriate), and then I can see the variables I have in the local environment, and how vt prints out the wrong variable name for the index i in the VGen i xs.

I tried to remove the call to reverse in value2term but the indices were just differently wrong.

I don't dare touch this further, because this seems like a fundamental thing in the GF compiler. Originally I just wanted to add another case in the glue function to match the tokens that were erroneously covered by "unsupported token gluing".

Take this minimum working example:

import PGF2
import Control.Exception
import qualified Data.Map as Map

main :: IO ()
main = do
  pgf <- readPGF "Feedback.pgf"
  let Just eng = Map.lookup "FeedbackEng" (languages pgf)
  onException (do
    let expr = mkApp (mkCId "BaseFeature") []
    let s = linearize eng expr
    putStrLn s
    ) (putStrLn "Caught exception")

where:

BaseFeature : Feature -> Feature -> ListFeature

So in my code I am building an invalid tree and expect to get an error. What currently happens is that my program crashes with the following:

pgf_cnc_resolve_app (pgf/linearizer.c:176): assertion failed
	n_args == gu_seq_length(papply->args)

Is there any way this can be thrown as a Haskell exception so that it can be handled appropriately?

reproducing the issue reported by @heatherleaf on https://groups.google.com/forum/#!topic/gf-dev/zYJn2dpaKeQ:

I've stumbled upon a problem with the cc command, for some RGL languages (e.g., English, German, all Scandinavian and Romance languages). Apparently it doesn't recognise when we hide functions from imports:

> i -retain -no-pmcfg english/LangEng.gf
> cc UttS (UseCl (TTAnt TPres ASimul) PPos (PredVP (DetCN (DetQuant DefArt NumSg) (UseN house_N)) (UseComp (CompAP (PositA big_A)))))
Warning: atomic term PPos
         conflict TenseX.PPos, GrammarEng.PPos
         given MarkHTMLX, CommonX, CatEng, MarkupEng, CommonX, CatEng,
               DocumentationEng, CommonX, CatEng, ConstructionEng, CommonX,
               CatEng, LexiconEng, CommonX, TenseX, CommonX, CatEng, IdiomEng,
               CommonX, CatEng, StructuralEng, CommonX, TextX, CommonX, CatEng,
               PhraseEng, CommonX, CatEng, ConjunctionEng, CommonX, CatEng,
               RelativeEng, CommonX, CatEng, QuestionEng, CommonX, CatEng,
               SentenceEng, CommonX, CatEng, NumeralEng, CommonX, CatEng,
               AdverbEng, CommonX, CatEng, AdjectiveEng, CommonX, CatEng, VerbEng,
               CommonX, CatEng, NounEng, GrammarEng, LangEng
missing record fields: p type of PPos
expected: {s : Str; p : ResEng.CPolarity}
inferred: {s : Str; lock_Pol : {}; p : ParamX.Polarity}

TenseX is imported in english/GrammarEng.gf, but PPos is hidden from the import. It seems like cc doesn't recognise that:

concrete GrammarEng of Grammar =
    (...),
    TenseX - [Pol,PPos,PNeg,SC,CAdv]
    ** open ResEng, Prelude in {
(...)
lin
  PPos = {s = [] ; p = CPos} ;
  PNeg = {s = [] ; p = CNeg True} ; -- contracted: don't
}

The same problem occurs for many languages, but not all. E.g., Finnish, Dutch, Afrikaans, Latin all work fine:

> i -retain -no-pmcfg finnish/LangFin.gf
> cc UttS (UseCl (TTAnt TPres ASimul) PPos (PredVP (DetCN (DetQuant DefArt NumSg) (UseN house_N)) (UseComp (CompAP (PositA big_A)))))
{s = "talo" ++ "on" ++ "suuri"; lock_S = <>; lock_Utt = <>}

personally, I'd love to see this fixed too because some grammars take very long to compile (PGF) or link (GFO), so we can't iterate as fast, and testing is slow.

When #26 is solved, one possible output format is a syntax-correct GF grammar. (But canonical -- no lambdas, no opers, etc)

This is an generalisation of what GF.Compile.ConcreteToHaskell does.

Originally reported by Roman Suzi in gf-dev.

When trying to extend a record produced by an overloaded oper, type inference fails:

resource TestRes = {
  oper
    Rec: Type = {
      a, b: Str ;
    } ;

    mkRec1: Str -> Rec = \a -> { a = a; b = a } ;
    mkRec2: Str -> Str -> Rec = \a,b -> { a = a; b = b } ;
    
    mkRec = overload {
      mkRec: Str -> Rec = mkRec1 ;
      mkRec: Str -> Str -> Rec = mkRec2 ;
    } ;
    
    r1: Rec = mkRec1 "A" ** {a = "X"} ; -- ok
    r2: Rec = mkRec2 "A" "B" ** {a = "X"} ; -- ok

    r3: Rec = mkRec "A" "B" ; -- ok
    r4: Rec = mkRec "A" "B" ** {a = "X"} ; -- fails
    r5: Rec =
      let r: Rec = mkRec "A" "B"
      in  r ** {a = "X"} ; -- ok
    r6: Rec = <mkRec "A" "B": Rec> ** {a = "X"} ; -- fails
}

Error for r4:

 TestRes.gf:19:
   Happened in operation r4
    no overload instance of TestRes.mkRec
    with value type {b : Str}
    for argument list
      Str Str
    among alternatives
      Str
      Str Str

Error for r6:

 TestRes.gf:23:
   Happened in operation r6
    no overload instance of TestRes.mkRec
    with value type {a : Str; b : Str}
    for argument list
      Str Str
    among alternatives
      Str
      Str Str

Minimal test grammar

abstract Test = {

cat S ;
fun testS : S ;
}

And here's a concrete syntax that causes an unsupported token gluing error unnecessarily.

concrete TestCnc of Test = open Prelude in {

lincat S = SS ;
lin testS = mkSS "test" ;

oper
  mkSS : Str -> SS ;
  mkSS str = case str of {
    x + "foo" => ss (x + "bar")
  } ;
}

Error

The error I get is this:

- compiling TestCnc.gf... gf: 8-10: In mkSS: unsupported token gluing: str + "bar"
CallStack (from HasCallStack):
  error, called at src/compiler/GF/Compile/Compute/ConcreteNew.hs:294:32 in gf-3.10.3-78vkYEvqjDa3tJH3g2tXyo:GF.Compile.Compute.ConcreteNew

Now I completely agree that the concrete syntax should cause an error, but just not about unsupported token gluing.

Minimal variations that don't cause the error

I found three ways to get rid of the misguided token gluing error:

a) Add a catch-all case after the x + "foo" case.

No errors anymore.

  mkSS str = case str of {
     x + "foo" => ss (x + "bar") ;
     x         => ss (x + "baz") } ;

b) Don't use the variable x on the right-hand side, but use the variable str instead.

This gives the expected error gf: Internal error in GeneratePMCFG: evalTerm ("test").

  mkSS str = case str of {
     x + "foo" => ss (str + "bar") } ;

c) Use the variable x on the RHS but don't glue the string "bar".

This gives the expected error gf: Internal error in GeneratePMCFG: evalTerm ("test").

  mkSS str = case str of {
     x + "foo" => ss x } ;

Link to grammars

The grammars are found in this gist: https://gist.github.com/inariksit/78a1929c4e98c5cc9d6ea693f7d0127f.

Hi,
if I have an abstract syntax like

abstract Equal = {
  cat Num;
      IsEqual Num Num ;
  fun
    z : Num ;
    s : Num -> Num ;
    equal : (n : Num) -> IsEqual n n ;
}

and generate trees of the category IsEqual with meta-variables, I get the following result as expected

Equal> gt -cat="IsEqual ? z"
equal z

but if I change the position of the question mark I get the following error:

Equal> gt -cat="IsEqual z ?"
src/runtime/haskell/PGF/TypeCheck.hs:(178,35)-(179,54): Non-exhaustive patterns in case

GF.Compiler.compileSourceFiles calls linkGrammars which calls exportPGF for printing additional output formats (e.g., Haskell, Javascript, SRGS, ...)

Sometimes we don't want to print the PGF, but instead the compiled canonical grammar. This is already done in cncs2haskell, but we want to be more generic. I.e., something like this:

compileSourceFiles opts fs = 
    do output@(_,(_,grammar)) <- batchCompile opts fs
       writeOutputs opts grammar
       unless (flag optStopAfterPhase opts == Compile) $
           linkGrammars opts output

Or, with some refactoring:

compileSourceFiles opts fs = 
    do output@(_,(_,grammar)) <- batchCompile opts fs
        when (flag optExportGrammar opts) $
            exportGrammar opts grammar
        unless (flag optStopAfterPhase opts == Compile) $
            pgf <- linkGrammars opts output
            when (flag optExportPGF opts) $
                exportPGF opts pgf

Most (all?) output formats can be described as a concrete syntax of a GF grammar.

This relates to issue #26 , and should make it simpler to add and modify different output formats.

Idea: write one GF grammar for canonical GF (ie what’s left after compilation and partial evaluation), and output formats such a Haskell, Javascript, Json, Yaml etc as concrete syntaxes. The same can be done for outputting PGF/PMCFG grammars. And possibly the same for speech grammars (which need some grammar transformations such as left-recursion elimination)

This would affect the modules GF.Compiler, GF.Compile.Export, and others. In fact, several modules can probably be removed (such as GF.Compile.PGFtoXxx)

One extra feature would be that we don’t have to recompile GF to add a new output format. So, anyone can create their own :)

This is not urgent by any means, but I'm bringing this up so that it's not buried in old commit messages.

The GF codebase includes a modified version of Haskell's Data.Binary from 2008. According to Grégoire's commit from 2010, this modification was necessary, because the binary representation of doubles in Data.Binary was (is?) not compliant with IEEE 754.

In the commit history, you can see a comment from 2013:

The standard binary package has improved efficiency and error handling [1], so
in the long run we should consider switching to it.

Is this still the case? The code from 2008 contains some unsafe IO operations (inlinePerformIO is deprecated and has a newer, more accurate name accursedUnutterablePerformIO). Later versions of Data.Binary don't contain accursedUnutterable…, at least not in those functions I checked.

Maybe more interesting, the implementation of the Get monad has changed, see current version of Binary vs. old version used in GF.

Ways to proceed

From least work to most work:

1. No real changes: Keep the modified Data.Binary from 2008 with all its curses, accursedUnutterablePerformIO is actually necessary in our code (for performance reasons?).
2. Keep the modified Data.Binary from 2008, but replace accursed unutterable functions with something safer, like unsafePerformIO.
3. If Data.Binary has fixed its representation of doubles and any other issues for which the custom module was used: Switch to the latest version of Data.Binary library. (+Write some proper property-based tests to make sure that the change doesn't break GF.)

I have a brand new MacOS Mojave installation, and the latest Haskell Platform (v 8.6.3). When I try to install GF, I get this:

$ cabal install gf
Warning: The install command is a part of the legacy v1 style of cabal usage.

Please switch to using either the new project style and the new-install
command or the legacy v1-install alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.

For more information, see: https://wiki.haskell.org/Cabal/NewBuild

Resolving dependencies...
In order, the following would be installed:
containers-0.5.11.0 (latest: 0.6.0.1) (via: Cabal-2.2.0.1 binary-0.8.6.0) (new
version)
network-2.6.3.6 (latest: 2.8.0.0) (via: gf-3.10 cgi-3001.3.0.3
httpd-shed-0.4.0.3) (new package)
network-uri-2.6.1.0 (via: gf-3.10 cgi-3001.3.0.3 httpd-shed-0.4.0.3) (new
package)
old-locale-1.0.0.7 (via: old-time-1.1.0.3) (new package)
parallel-3.2.2.0 (via: gf-3.10) (new package)
random-1.1 (via: gf-3.10) (new package)
stringsearch-0.3.6.6 (via: multipart-0.1.3) (new package)
syb-0.7 (via: json-0.9.3) (new package)
transformers-compat-0.6.2 (via: exceptions-0.10.0) (new package)
utf8-string-1.0.1.1 (via: gf-3.10) (new package)
binary-0.8.6.0 (latest: 0.10.0.0) (via: Cabal-2.2.0.1 text-1.2.3.1)
(reinstall) (changes: containers-0.6.0.1 removed)
httpd-shed-0.4.0.3 (via: gf-3.10) (new package)
old-time-1.1.0.3 (via: time-compat-0.1.0.3) (new package)
multipart-0.1.3 (via: cgi-3001.3.0.3) (new package)
json-0.9.3 (via: gf-3.10) (new package)
exceptions-0.10.0 (via: gf-3.10 cgi-3001.3.0.3) (new package)
text-1.2.3.1 (via: Cabal-2.2.0.1 parsec-3.1.13.0 json-0.9.3) (reinstall)
(changes: binary-0.8.6.0 removed)
time-compat-0.1.0.3 (via: gf-3.10) (new package)
cgi-3001.3.0.3 (via: gf-3.10) (new package)
parsec-3.1.13.0 (via: Cabal-2.2.0.1 cgi-3001.3.0.3 json-0.9.3 multipart-0.1.3
network-uri-2.6.1.0) (reinstall) (changes: text-1.2.3.1 removed)
Cabal-2.2.0.1 (latest: 2.4.1.0) (via: gf-3.10) (new version)
gf-3.10 (new package)
cabal: The following packages are likely to be broken by the reinstalls:
Cabal-2.4.0.1
libiserv-8.6.3
ghci-8.6.3
ghc-8.6.3
ghc-boot-8.6.3
Use --force-reinstalls if you want to install anyway.

But when I clone the github repo, I can cabal install from the source directory.

I have started working on updating the GF website. The work is in the branch new-website. My modest goals are:

1. Cleaning it up visually
2. Making information easier to find

This issue can serve as a discussion thread for these updates, for anyone who's interested.
Intention is go live with the updated site in conjunction with the upcoming GF release on November 28.

Other notes

• Even though I'm not a fan of txt2tags, so much content is already written in this format and it is not trivial to losslessly convert it to something else like Markdown. On the other hand, supporting Markdown for new content should be easy enough.
• There are many URLs under the grammaticalframework.org domain to things which are not in this repository. Moving hosting (e.g. to GitHub pages) would break these, and thus this is not a consideration.
• I don't see much need to start using a complex site generation like Jekyll.

When loading the following Grammar

abstract TestAbs = {
  cat
    Temp ; Pol ; Cl ; S ; 
  fun
    UseCl  : Temp -> Pol -> Cl -> S ;
}

  param Order   = Ord | Subord | Quest;
  lincat
    Cl = {s : Temp => Bool => Order => Str};
    S = {s : Order => Str};
    Pol = {s : Str ; b : Bool};
    Temp = { t : Bool * Bool } ;
  lin
    UseCl temp pol cl = { --Temp -> Pol -> Cl -> S ;  
      s = \\o => case o of {
	Ord => cl.s ! lin Temp temp ! pol.b ! Ord;
	_ => "Baz" 
	};
      };
}

this error occurs
compiling Test.gf... src/compiler/GF/Compile/GeneratePMCFG.hs:(430,42)-(431,76): Non-exhaustive patterns in case

the LICENSE file in this repo is outdated after the repository splits, and gf-rgl ended up without a license in its repo!

I wonder why alex and happy are not listed as dependencies on gf.cabal? I'm probably unaware of possible drawbacks, but it seems nice to have cabal install them for me.

This text is found in the tutorial, Lesson 7, Using the JavaScript grammar:

To perform parsing and linearization, the run-time library gflib.js is used. It is included in GF/lib/javascript/, together with some other JavaScript and HTML files; these files can be used as templates for building applications.

GF/lib/javascript/ doesn't exist anymore. Does gflib.js still exist, and if so, where is it found?

An example of usage is translator.html, which is in fact initialized with a pointer to the Food grammar, so that it provides translation between the English and Italian grammars:

The link to translator.html on the page doesn't work.

I wonder if you would accept a PR which integrates https://cibuildwheel.readthedocs.io/en/stable/setup/ to automatically build wheel packages for Python. This would mean that people can install from PyPI without having to download gf-core and build their own packages.

I can see there is already a Travis build running, it could be added there, or run through Github Actions. I assume Travis would be preferred?

today I came across the following error message:

_gf_preproc.tmp:768:1:
   conflicting information in module ParadigmsEng             
    oper mkV2A = overload {mkV2A : V -> V2A; mkV2A : V -> Prep -> V2A;
                           mkV2A : V -> Prep -> Prep -> V2A} ;      
and                                                                 
    oper mkV2A = overload {                                             
                   mkV2A : V -> V2A = \v -> lin V2A (dirdirV3 v) ;
                   mkV2A : V -> Prep -> V2A
                         = \v,p -> lin V2A (dirV3 v p) ;
                   mkV2A : V -> Prep -> Prep -> V2A
                         = \v,p1,p2 -> lin V2A (prepPrepV3 v p1 p2) ;
                 } ;

the problem was in the first judgement where I put a = where a colon should be, but that's not what the message tells me. of course, when I comment the second judgement I get the real error:

src/english/ParadigmsEng.gf:324-328:
  Happened in the renaming of mkV2A
   constant not found: overload
   given Predef, Predef, CatEng, ResEng, MorphoEng, Prelude,
         ParadigmsEng

which is still not that direct now that I think of it, but oh well.

I understand that this is not high priority and that it's probably not an easy fix because these errors are found in different steps of the processing, but I felt it it deserved the reporting anyway!

Here is a problem with how GF interprets variants -- I guess it's a bug somewhere.

Variants> l -all adjNPv2 good (mkNP king)
bonus rex
rex bonus

Variants> l -all adjNPv1 good (mkNP king)
bonus rex
rex bonus

Variants> l -all mkNP (adjCNv1 good king)
bonus rex
rex bonus

Variants> l -all mkNP (adjCNv2 good king)
bonus rex
bonus rex bonus
rex
rex bonus

The only difference between adjCNv1 and adjCNv2 is that in v2 the variants{...} is inlined in the function, but in v1 it is in the argument (ap.position) instead. The other two adjNPv1/2 don't show the same problem.

best,
Peter

PS. Here is the grammar:

abstract Variants = {
flags startcat = NP;
cat NP; CN; AP;
fun
 king : CN;
 good : AP;
 mkNP : CN -> NP;
 adjCNv1, adjCNv2 : AP -> CN -> CN;
 adjNPv1, adjNPv2 : AP -> NP -> NP;
}

concrete VariantsCnc of Variants = {
lincat
 NP = {s : Str};
 CN = {s, before, after : Str};
 AP = {s : Str; position : Position};
param
 Position = Before | After ;
lin
 king = {s = "rex"; before = []; after = []};
 good = {s = "bonus"; position = variants{Before ; After}};

 mkNP cn = {s = cn.before ++ cn.s ++ cn.after};

 adjNPv1 ap np = 
   let pos = ap.position in
   {s = case pos of { Before => ap.s ++ np.s ;
                      After  => np.s ++ ap.s };
   };

 adjNPv2 ap np = 
   let pos = variants{Before ; After} in
   {s = case pos of { Before => ap.s ++ np.s ;
                      After  => np.s ++ ap.s };
   };

 adjCNv1 ap cn = 
   let pos = ap.position in
   {s = cn.s;
    before = case pos of { Before => ap.s ++ cn.before ; After => cn.after };
    after  = case pos of { Before => cn.before ; After => ap.s ++ cn.after };
   };

 -- this generates an error, because the two occurrences of pos will not be in sync:
 adjCNv2 ap cn = 
   let pos = variants{Before ; After} in
   {s = cn.s;
    before = case pos of { Before => ap.s ++ cn.before ; After => cn.after };
    after  = case pos of { Before => cn.before ; After => ap.s ++ cn.after };
   };

}

I fail when I try to set up the web interface for the phrasebook following these instructions.

After installing the four third party libraries (cgi >= 3001.1.7.0, fastcgi >= 3001.0.2.1, json >= 0.3.3, utf8-string >= 0.3.1.1 ) stated under the section Compilation and running the command

$ runghc Setup.hs configure

I get the following warnings:

Configuring gf-server-1.0...
Warning: 'license-file: ../../LICENSE' is a relative path outside of the
source tree. This will not work when generating a tarball with 'sdist'.
Warning: This package indirectly depends on multiple versions of the same
package. This is highly likely to cause a compile failure.
package gf-server-1.0 requires network-2.5.0.0
package httpd-shed-0.4.0.3 requires network-2.6.3.6
package gf-3.9 requires network-2.6.3.6
package cgi-3001.3.0.2 requires network-2.6.3.6

Is there anything that can be done about this?

Note: this is not an issue per se, but instead background information about the semantics of variants. I will reference this issue in some later issues which contain actual suggestions.

There are two possible formal semantics for the disjunction (called variants in GF). In sections 5.2 and 5.3 of my thesis I call them "intensional" and "extensional" disjunction (https://gup.ub.gu.se/file/207628):

1. Intensional disjunction

Intensional disjunction is formally just a shorthand for defining a number of GF functions. This means that we can "push out" variants to the top-level of a linearization. So this definition:

lin f = {a = "a"; b = variants{"b1" ; "b2"}}

becomes the same as this:

lin f = variants{ {a="a"; b="b1"} ; {a="a"; b="b2"} }

which is semantically equivalent to this:

lin f1 = {a="a"; b="b1"}
lin f2 = {a="a"; b="b2"}

This is the disjunction that is implemented in GF since a couple of years.

• The main advantage is of course that the current parser works for this semantics.

• One drawback is that the size of the grammar can explode - for each disjunction the number of linearizations for that function doubles, so if you have a function with 10 constituents (i.e., a record with 10 labels), and every constituent contains one disjunction (i.e., variants{s1;s2}), then the function will have 2¹⁰ = 1000 different linearizations. (With 20 record label, the function will have 2²⁰ = 1 million linearizations)

• Another drawback is that variants{} becomes practically useless - as soon as you put it anywhere in you linearization, the function will have no linearizations at all. This is because {a = "a"; b = variants{}} is the same as variants{} according to the intensional semantics.

2. Extensional disjunction

Extensional disjunction is set union. This requires that GF linearizes syntax trees to sets of linearizations. The effect of this is that disjunctions cannot be pushed to the top-level (at least not in all cases). The formal difference with intensional disjunction is that reduplication behaves differently -- read more in my thesis if you want to know why.

• The main drawback is that GF's current parsing algorithm (implemented by Krasimir) cannot handle extensional disjunction correctly. I don't know how difficult it is to change.

• But the main advantage is that variants{} becomes useful: Formally, under this semantics variants{} is equivalent to a dummy element of the correct type.

Actually, this was the semantics that was implemented in GF until some years ago.

$ gf Test.gf
                              
         *  *  *              
      *           *           
    *               *         
   *                          
   *                          
   *        * * * * * *       
   *        *         *       
    *       * * * *  *        
      *     *      *          
         *  *  *              
                              
This is GF version 3.10.4. 
Built on darwin/x86_64 with ghc-8.2, flags: interrupt server c-runtime
License: see help -license.   

linking ... OK

Languages:
Test> r
no import in history

Surely the module I specified on the command line should count as an import?
This has driven me mad for years, so I'm finally fixing it creating an issue about it.

my GF_LIB_PATH is at /home/bruno/.cabal/share/x86_64-linux-ghc-8.2.1/gf-3.9/lib/, and this is were the tags files are being written:

$ gf --tags MatrixEng.gf
write file /home/bruno/.cabal/share/x86_64-linux-ghc-8.2.1/gf-3.9/lib/alltenses/Common.gf-tags
[...]
write file /home/bruno/.cabal/share/x86_64-linux-ghc-8.2.1/gf-3.9/lib/alltenses/MarkupEng.gf-tags
write file /home/bruno/sites/gf/matrix/MatrixEng.gf-tags

however if I check MatrixEng.gf-tags contents I have things like:

A	indir	ConstructionEng		/home/bruno/sites/gf/rgl/src/english/CatEng.gf-tags

which point to where my clone of the gf-rgl repository is, and where there are no tags files. any idea of why this could be happening? I have no compiler pragma pointing to my clone.