ibm / nl2ltl Goto Github PK

View Code? Open in Web Editor NEW

56.0 5.0 10.0 43.68 MB

Natural Language (NL) to Linear Temporal Logic (LTL)

License: MIT License

Python 100.00%

declare ibm-research-ai linear-temporal-logic ltl ltlf nautral-language-processing

nl2ltl's Introduction

NL 2 LTL

NL2LTL is an interface to translate natural language (NL) utterances to linear temporal logic (LTL) formulas.

🏆 NL2LTL won the People's Choice Best System Demonstration Award Runner-Up in the ICAPS 2023 System Demonstration Track in Prague. Read more about it here.

Installation

from PyPI:

pip install nl2ltl

from source (main branch):

pip install git+https://github.com/IBM/nl2ltl.git

or clone the repository and install the package:

git clone https://github.com/IBM/nl2ltl.git
cd nl2ltl
pip install -e .

Quickstart

Once you have installed all dependencies you are ready to go with:

from nl2ltl import translate
from nl2ltl.engines.gpt.core import GPTEngine, Models
from nl2ltl.filters.simple_filters import BasicFilter
from nl2ltl.engines.utils import pretty

engine = GPTEngine()
filter = BasicFilter()
utterance = "Eventually send me a Slack after receiving a Gmail"

ltlf_formulas = translate(utterance, engine, filter)
pretty(ltlf_formulas)

The translate function takes a natural language utterance, an engine and an option filter, and outputs the best matching pylogics LTL formulas.

NOTE: Before using the NL2LTL translation function, depending on the engine you want to use, make sure all preconditions for such an engine are met. For instance, Rasa requires a .tar.gz format trained model in the models/ folder to run. To train the model use the available NL2LTL train(...) API.

NLU Engines

GPT-3.x large language models
GPT-4 large language model
Rasa intents/entities classifier (to use Rasa, please install it with pip install -e ".[rasa]")
Watson Assistant intents/entities classifier -- Planned

NOTE: To use OpenAI GPT models don't forget to add the OPEN_API_KEY environment variable with:

export OPENAI_API_KEY=your_api_key

Write your own Engine

You can easily write your own engine (i.e., intents/entities classifier, language model, etc.) by implementing the Engine interface:

from nl2ltl.engines.base import Engine
from pylogics.syntax.base import Formula

class MyEngine(Engine):

    def translate(self, utterance: str, filtering: Filter) -> Dict[Formula, float]:
        """From NL to LTL."""

Then, use it as a parameter in the main entry point:

my_engine = MyEngine()
ltl_formulas = translate(utterance, engine=my_engine)

Write your own Filter

You can easily write your own filtering algorithm by implementing the Filter interface:

from nl2ltl.filters.base import Filter
from pylogics.syntax.base import Formula

class MyFilter(Filter):

    def enforce(
        self, output: Dict[Formula, float], entities: Dict[str, float], **kwargs
    ) -> Dict[Formula, float]:
    """Filtering algorithm."""

Then, use it as a parameter in the main entry point:

my_engine = MyEngine()
my_filter = MyFilter()
ltl_formulas = translate(utterance, engine=my_engine, filter=my_filter)

Development

Contributions are welcome! Here's how to set up the development environment:

set up your preferred virtualenv environment
clone the repo: git clone https://github.com/IBM/nl2ltl.git && cd nl2ltl
install dependencies: pip install -e .
install dev dependencies: pip install -e ".[dev]"
install pre-commit: pre-commit install
sign-off your commits using the -s flag in the commit message to be compliant with the DCO

Tests

To run tests: tox

To run the code tests only: tox -e py310

Docs

To build the docs: mkdocs build

To view documentation in a browser: mkdocs serve and then go to http://localhost:8000

Citing

@inproceedings{icaps2023fc,
  author       = {Francesco Fuggitti and  Tathagata Chakraborti},
  title        = {{NL2LTL} -- A Python Package for Converting Natural Language ({NL}) Instructions to Linear Temporal Logic ({LTL}) Formulas},
  booktitle    = {{ICAPS}},
  year         = {2023},
  note         = {Best System Demonstration Award Runner-Up.},
  url_code     = {https://github.com/IBM/nl2ltl},
}

and

@inproceedings{aaai2023fc,
  author       = {Francesco Fuggitti and  Tathagata Chakraborti},
  title        = {{NL2LTL} -- A Python Package for Converting Natural Language ({NL}) Instructions to Linear Temporal Logic ({LTL}) Formulas},
  booktitle    = {{AAAI}},
  year         = {2023},
  note         = {System Demonstration.},
  url_code     = {https://github.com/IBM/nl2ltl},
}

nl2ltl's People

Contributors

Stargazers

Watchers

Forkers

ramonpereira suryanshuts alexdesiqueira yagouus jeevjohn tabithako maxvidgof marmikreal anastasiaslobodyanik samyaza-geek

nl2ltl's Issues

Error with integration tests

Report: https://github.com/IBM/nl2ltl/actions/runs/3064589654.

Integration with GOFA

Illustration of

Integration with Plan4Past.
Application in an industry use case.

Integration with Open AI API

Whitepaper

For arXiv (after the AAAI Demo deadline).

Bug on Greedy filter

The set inclusion checks within the greedy filter do not work properly.

Cannot run the QuickStart

from nl2ltl import translate
from nl2ltl.engines.rasa.core import RasaEngine
from nl2ltl.engines.gpt.core import openai
from nl2ltl.filters.simple_filters import BasicFilter
from nl2ltl.engines.utils import pretty

engine = RasaEngine()
#engine = openai.Engine()
filter = BasicFilter()
utterance = "Eventually send me a Slack after receiving a Gmail"

ltlf_formulas = translate(utterance, engine, filter)
pretty(ltlf_formulas)

ImportError Traceback (most recent call last)
File ~/.local/lib/python3.9/site-packages/rasa/engine/graph.py:112, in GraphSchema.from_dict(cls, serialized_graph_schema)
109 try:
110 serialized_node[
111 "uses"
--> 112 ] = rasa.shared.utils.common.class_from_module_path(
113 serialized_node["uses"]
114 )
116 resource = serialized_node["resource"]

File ~/.local/lib/python3.9/site-packages/rasa/shared/utils/common.py:36, in class_from_module_path(module_path, lookup_path)
35 module_name, _, class_name = module_path.rpartition(".")
---> 36 m = importlib.import_module(module_name)
37 klass = getattr(m, class_name, None)

File /usr/lib/python3.9/importlib/init.py:127, in import_module(name, package)
126 level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File :1030, in _gcd_import(name, package, level)

File :1007, in find_and_load(name, import)

File :986, in find_and_load_unlocked(name, import)

File :680, in _load_unlocked(spec)

File :850, in exec_module(self, module)

File :228, in _call_with_frames_removed(f, *args, **kwds)

File ~/.local/lib/python3.9/site-packages/rasa/graph_components/validators/default_recipe_validator.py:12
11 from rasa.nlu.extractors.regex_entity_extractor import RegexEntityExtractor
---> 12 from rasa.nlu.extractors.crf_entity_extractor import (
13 CRFEntityExtractor,
14 CRFEntityExtractorOptions,
15 )
16 from rasa.nlu.extractors.entity_synonyms import EntitySynonymMapper

File ~/.local/lib/python3.9/site-packages/rasa/nlu/extractors/crf_entity_extractor.py:18
17 from rasa.engine.storage.storage import ModelStorage
---> 18 from rasa.nlu.test import determine_token_labels
19 from rasa.nlu.tokenizers.spacy_tokenizer import POS_TAG_KEY

File ~/.local/lib/python3.9/site-packages/rasa/nlu/test.py:34
33 import rasa.shared.utils.io
---> 34 import rasa.utils.plotting as plot_utils
35 import rasa.utils.io as io_utils

File ~/.local/lib/python3.9/site-packages/rasa/utils/plotting.py:8
7 from typing import Any, Callable, List, Optional, Text, TypeVar, Union, Tuple
----> 8 import matplotlib
9 from matplotlib.ticker import FormatStrFormatter

File ~/.local/lib/python3.9/site-packages/matplotlib/init.py:109
107 # cbook must import matplotlib only within function
108 # definitions, so it is safe to import from it here.
--> 109 from . import _api, _version, cbook, docstring, rcsetup
110 from matplotlib.cbook import MatplotlibDeprecationWarning, sanitize_sequence

File ~/.local/lib/python3.9/site-packages/matplotlib/rcsetup.py:27
26 from matplotlib.cbook import ls_mapper
---> 27 from matplotlib.colors import Colormap, is_color_like
28 from matplotlib.fontconfig_pattern import parse_fontconfig_pattern

File ~/.local/lib/python3.9/site-packages/matplotlib/colors.py:51
50 import re
---> 51 from PIL import Image
52 from PIL.PngImagePlugin import PngInfo

File /usr/lib/python3/dist-packages/PIL/Image.py:89
83 try:
84 # If the _imaging C module is not present, Pillow will not load.
85 # Note that other modules should not refer to _imaging directly;
86 # import Image and use the Image.core variable instead.
87 # Also note that Image.core is not a publicly documented interface,
88 # and should be considered private and subject to change.
---> 89 from . import _imaging as core
91 if version != getattr(core, "PILLOW_VERSION", None):

ImportError: cannot import name '_imaging' from 'PIL' (/usr/lib/python3/dist-packages/PIL/init.py)

The above exception was the direct cause of the following exception:

GraphSchemaException Traceback (most recent call last)
Cell In[3], line 7
4 from nl2ltl.filters.simple_filters import BasicFilter
5 from nl2ltl.engines.utils import pretty
----> 7 engine = RasaEngine()
8 #engine = openai.Engine()
9 filter = BasicFilter()

File ~/Documents/nl2ltl/nl2ltl/engines/rasa/core.py:42, in RasaEngine.init(self, model)
37 def init(
38 self,
39 model: Path = None,
40 ):
41 """Rasa NLU Engine initialization."""
---> 42 self._load_model(model)
44 self._check_consistency()

File ~/Documents/nl2ltl/nl2ltl/engines/rasa/core.py:50, in RasaEngine._load_model(self, model)
48 self.agent = Agent.load(model)
49 else:
---> 50 self.agent = Agent.load(_get_latest_model(path=MODEL_OUTPUT_PATH))

File ~/.local/lib/python3.9/site-packages/rasa/core/agent.py:345, in Agent.load(cls, model_path, domain, generator, tracker_store, lock_store, action_endpoint, fingerprint, model_server, remote_storage, http_interpreter)
333 """Constructs a new agent and loads the processer and model."""
334 agent = Agent(
335 domain=domain,
336 generator=generator,
(...)
343 http_interpreter=http_interpreter,
344 )
--> 345 agent.load_model(model_path=model_path, fingerprint=fingerprint)
346 return agent

File ~/.local/lib/python3.9/site-packages/rasa/core/agent.py:352, in Agent.load_model(self, model_path, fingerprint)
348 def load_model(
349 self, model_path: Union[Text, Path], fingerprint: Optional[Text] = None
350 ) -> None:
351 """Loads the agent's model and processor given a new model path."""
--> 352 self.processor = MessageProcessor(
353 model_path=model_path,
354 tracker_store=self.tracker_store,
355 lock_store=self.lock_store,
356 action_endpoint=self.action_endpoint,
357 generator=self.nlg,
358 http_interpreter=self.http_interpreter,
359 )
360 self.domain = self.processor.domain
362 self._set_fingerprint(fingerprint)

File ~/.local/lib/python3.9/site-packages/rasa/core/processor.py:105, in MessageProcessor.init(self, model_path, tracker_store, lock_store, generator, action_endpoint, max_number_of_predictions, on_circuit_break, http_interpreter)
103 self.on_circuit_break = on_circuit_break
104 self.action_endpoint = action_endpoint
--> 105 self.model_filename, self.model_metadata, self.graph_runner = self._load_model(
106 model_path
107 )
109 if self.model_metadata.assistant_id is None:
110 rasa.shared.utils.io.raise_warning(
111 f"The model metadata does not contain a value for the "
112 f"'{ASSISTANT_ID_KEY}' attribute. Check that 'config.yml' "
(...)
116 UserWarning,
117 )

File ~/.local/lib/python3.9/site-packages/rasa/core/processor.py:142, in MessageProcessor._load_model(model_path)
140 with TempDirectoryPath(get_temp_dir_name()) as temporary_directory:
141 try:
--> 142 metadata, runner = loader.load_predict_graph_runner(
143 Path(temporary_directory),
144 Path(model_tar),
145 LocalModelStorage,
146 DaskGraphRunner,
147 )
148 return os.path.basename(model_tar), metadata, runner
149 except tarfile.ReadError:

File ~/.local/lib/python3.9/site-packages/rasa/engine/loader.py:26, in load_predict_graph_runner(storage_path, model_archive_path, model_storage_class, graph_runner_class)
9 def load_predict_graph_runner(
10 storage_path: Path,
11 model_archive_path: Path,
12 model_storage_class: Type[ModelStorage],
13 graph_runner_class: Type[GraphRunner],
14 ) -> Tuple[ModelMetadata, GraphRunner]:
15 """Loads a model from an archive and creates the prediction graph runner.
16
17 Args:
(...)
24 A tuple containing the model metadata and the prediction graph runner.
25 """
---> 26 model_storage, model_metadata = model_storage_class.from_model_archive(
27 storage_path=storage_path, model_archive_path=model_archive_path
28 )
29 runner = graph_runner_class.create(
30 graph_schema=model_metadata.predict_schema,
31 model_storage=model_storage,
(...)
34 ),
35 )
36 return model_metadata, runner

File ~/.local/lib/python3.9/site-packages/rasa/engine/storage/local_model_storage.py:95, in LocalModelStorage.from_model_archive(cls, storage_path, model_archive_path)
89 logger.debug(f"Extracted model to '{temporary_directory_path}'.")
91 cls._initialize_model_storage_from_model_archive(
92 temporary_directory_path, storage_path
93 )
---> 95 metadata = cls._load_metadata(temporary_directory_path)
97 return (cls(storage_path), metadata)

File ~/.local/lib/python3.9/site-packages/rasa/engine/storage/local_model_storage.py:154, in LocalModelStorage._load_metadata(directory)
148 @staticmethod
149 def _load_metadata(directory: Path) -> ModelMetadata:
150 serialized_metadata = rasa.shared.utils.io.read_json_file(
151 directory / MODEL_ARCHIVE_METADATA_FILE
152 )
--> 154 return ModelMetadata.from_dict(serialized_metadata)

File ~/.local/lib/python3.9/site-packages/rasa/engine/storage/storage.py:194, in ModelMetadata.from_dict(cls, serialized)
178 """Loads ModelMetadata which has been serialized using metadata.as_dict().
179
180 Args:
(...)
184 Instantiated ModelMetadata.
185 """
186 from rasa.engine.graph import GraphSchema
188 return ModelMetadata(
189 trained_at=datetime.fromisoformat(serialized["trained_at"]),
190 rasa_open_source_version=serialized["rasa_open_source_version"],
191 model_id=serialized["model_id"],
192 assistant_id=serialized.get("assistant_id"),
193 domain=Domain.from_dict(serialized["domain"]),
--> 194 train_schema=GraphSchema.from_dict(serialized["train_schema"]),
195 predict_schema=GraphSchema.from_dict(serialized["predict_schema"]),
196 training_type=TrainingType(serialized["training_type"]),
197 project_fingerprint=serialized["project_fingerprint"],
198 core_target=serialized["core_target"],
199 nlu_target=serialized["nlu_target"],
200 language=serialized["language"],
201 # optional, since introduced later
202 spaces=serialized.get("spaces"),
203 )

File ~/.local/lib/python3.9/site-packages/rasa/engine/graph.py:121, in GraphSchema.from_dict(cls, serialized_graph_schema)
118 serialized_node["resource"] = Resource(**resource)
120 except ImportError as e:
--> 121 raise GraphSchemaException(
122 "Error deserializing graph schema. Can't "
123 "find class for graph component type "
124 f"'{serialized_node['uses']}'."
125 ) from e
127 nodes[node_name] = SchemaNode(**serialized_node)
129 return GraphSchema(nodes)

GraphSchemaException: Error deserializing graph schema. Can't find class for graph component type 'rasa.graph_components.validators.default_recipe_validator.DefaultV1RecipeValidator'.

Badges

Python versions
PyPI version
Build
Coverage

Issue with Rasa Engine

Hi,

I have some problems with using the Rasa Engine for my use case. The error is "ModelNotFound: Model None can not be loaded."

Can you please help me with resolving this issue ?

The Note in the readme file says that a tar file is required for this. But I am not able to fix this issue.

Planutils Integration

Would love to see a wrapped version that lets us do something like this:

$ planutils install nl2ltl
$ nl2ltl --input "At some point you need to either submit a paper or give up." --format spot

Formats could be spot or pddl or whatever. Example of the latter would be...

(:goal <>( (have_image Phenomenon4 thermograph0) && (have_image Star5 thermograph0) && (have_image Phenomenon6 thermograph0))))

...as found in this work.

Scary list of warnings

/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/future/standard_library/__init__.py:65: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
  'nearest': pil_image.NEAREST,
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  'bilinear': pil_image.BILINEAR,
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
  'bicubic': pil_image.BICUBIC,
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
  if hasattr(pil_image, 'HAMMING'):
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
  if hasattr(pil_image, 'BOX'):
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
  if hasattr(pil_image, 'LANCZOS'):
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/matplotlib/__init__.py:169: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if LooseVersion(module.__version__) < minver:
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  other = LooseVersion(other)
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/tensorflow_addons/utils/ensure_tf_install.py:47: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  min_version = LooseVersion(INCLUSIVE_MIN_TF_VERSION)
2022-09-22 15:27:00.959324: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:448: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradients/cond_grad/gradients/cond/GatherV2_grad/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradients/cond_grad/gradients/cond/GatherV2_grad/Reshape:0", shape=(None,), dtype=float32), dense_shape=Tensor("gradients/cond_grad/gradients/cond/GatherV2_grad/Cast:0", shape=(1,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
  warnings.warn(
/opt/anaconda3/envs/nl2ltl/lib/python3.8/site-packages/rasa/utils/train_utils.py:528: UserWarning: constrain_similarities is set to `False`. It is recommended to set it to `True` when using cross-entropy loss.
  rasa.shared.utils.io.raise_warning(
======================================================================================================================================================
Declare Template: (ChainResponse Slack Gmail)
English meaning:  Every time activity Slack happens, it must be directly followed by activity Gmail (activity Gmail can also follow other activities).
Confidence:       0.9999997615814209

Declare Template: (ExistenceTwo Slack)
English meaning:  Slack will happen at least twice.
Confidence:       1.0441302578101386e-07

Declare Template: (RespondedExistence Slack Gmail)
English meaning:  If Slack happens at least once then Gmail has to happen or happened before Slack.
Confidence:       6.342362723898987e-08

Declare Template: (Response Slack Gmail)
English meaning:  Whenever activity Slack happens, activity Gmail has to happen eventually afterward.
Confidence:       4.357292482382036e-08

Declare Template: (Existence Slack)
English meaning:  Eventually, Slack will happen.
Confidence:       4.138687170751609e-09

Error getting started

I have downloaded rsa model. created a directory named 'models' put the rsa model in it.
When I run the sample code provided I get the following error:
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
I choose the open-api engine and and export the api-key and try run the code, I get the following error:

File "C:\Users\Lenovo\.conda\envs\nl2l\lib\site-packages\openai\openai_object.py", line 59, in __getattr__
    return self[k]
KeyError: 'translate'

My Code

from nl2ltl import translate
from nl2ltl.engines.rasa.core import RasaEngine
from nl2ltl.engines.gpt.core import openai
from nl2ltl.filters.simple_filters import BasicFilter
from nl2ltl.engines.utils import pretty

# engine = RasaEngine()
engine = openai.Engine()
filter = BasicFilter()
utterance = "Eventually send me a Slack after receiving a Gmail"

ltlf_formulas = translate(utterance, engine, filter)
pretty(ltlf_formulas)

detailed error files are attached
error-open-api-engine.txt
error-rsa-engine.txt

AAAI Demo Submission

Call: https://aaai.org/Conferences/AAAI-23/demonstration-program