node-search-demo's Introduction

Full-stack Node.js & Typescript Demo

Usage

Create the bridge network : docker network create --driver=bridge --attachable --internal=false default-bridge-network
Start Elastic EnterpriseSearch for initial configuration : docker-compose up enterprisesearch
Go to http://localhost:3002 and follow the onboarding by creating an engine name, use model-search-demo by default
Copy the auto-generated keys : one public for the front, one private for the back, go to http://localhost:3002/as#/credentials
Edit the .env file with environment variables for REACT_APP_ENTERPRISE_SEARCH_PUBLIC_KEY and ENTERPRISE_SEARCH_PRIVATE_KEY. Set also ENTERPRISE_SEARCH_INDEx of you used another name than model-search-demo
Start the front and back with docker-compose up
Indexation will immediately start, wait a bit for results to appear
Go to http://localhost:3000
You can also check the search engine at http://localhost:3002/as#/engines/model-search-demo/documents/

Data Backend

I used App Search from Elastic's Enterprise Search built on top of ElasticSearch, because it's easier to bootstrap a simple search engine demo UI with opensource Elastic's Search-UI React components.
In production, I'd rather build on top of ElasticSearch and develop our own UI, to have fine tuning control over the index and more complex search queries.

Real-world architecture VS. demo architecture

I 'd use Serverless function to process git changes and index them. For example, AWS Lambda or Kubernetes+OpenFaas as a distributed asynchronous serverless functions.
Local development environment for OpenFass/Kubernetes is easy to install with microk8s, minikube or k3s (docs: https://docs.openfaas.com/deployment/kubernetes/), and is a powerfull tool for scaling. AWS Lambda with serverless official framework is another powerfull infrastructure.
I'd reuse the function created here to index the latest README.md text of each git repository from https://huggingface.co from an event hooked by another process described below
I'd create another watcher process, that loops over the repositories regularly, and calls git diff-index --name-only HEAD to see if the README.md file has changed in order to call the event (SNS or OpenFaas Events) that triggers the async function.
Other things that have been ruled out of this demo for the sake of simplicity, like removing deleted models.

node-search-demo's People

Contributors

Watchers

node-search-demo's Issues

Error while running `docker-compose up enterprisesearch`

I tried to follow the readme to run the demo, but I get an error at the first step, here is the stack trace:

Starting elishowk_elasticsearch_1 ... done
Starting elishowk_enterprisesearch_1 ... done
Attaching to elishowk_enterprisesearch_1
enterprisesearch_1  | Custom Enterprise Search configuration file detected, not overwriting it (any settings passed via environment will be ignored)
enterprisesearch_1  | Found java executable in PATH
enterprisesearch_1  | Java version detected: 1.8.0_252 (major version: 8)
enterprisesearch_1  | Enterprise Search is starting...
enterprisesearch_1  | *** [DEPRECATION WARNING] The setting '#/ent_search/auth/source' is deprecated and will be removed in version '8.0.0'. Please use the new auth config format ent_search.auth.<auth_name>.source.
enterprisesearch_1  | Unexpected exception while running Enterprise Search:
enterprisesearch_1  | Error: elasticsearch: Name or service not known at /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/faraday/adapter/manticore.rb:88:in `block in call'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/manticore/response.rb:79:in `call'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/faraday/adapter/manticore.rb:92:in `call'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/lib/middleware/request_logging_middleware.class:25:in `call'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/faraday-0.17.3/lib/faraday/rack_builder.rb:143:in `build_response'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/faraday-0.17.3/lib/faraday/connection.rb:387:in `run_request'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/faraday-0.17.3/lib/faraday/connection.rb:138:in `get'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:161:in `check_es_connection_with_retries!'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:33:in `check_es_connection!'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:18:in `block in run!'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/tagged_logging.rb:68:in `block in tagged'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/tagged_logging.rb:26:in `tagged'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/tagged_logging.rb:68:in `tagged'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:17:in `run!'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:13:in `run!'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:256:in `configure_elasticsearch!'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:236:in `configure!'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/config/application.class:18:in `<main>'
enterprisesearch_1  |   org/jruby/RubyKernel.java:1016:in `load'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/dependencies.rb:268:in `block in load'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/dependencies.rb:240:in `load_dependency'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/dependencies.rb:268:in `load'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/config/application.rb:1:in `<main>'
enterprisesearch_1  |   org/jruby/RubyKernel.java:974:in `require'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/dependencies.rb:274:in `block in require'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/dependencies.rb:240:in `load_dependency'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/gems/gems/activesupport-4.2.11.3/lib/active_support/dependencies.rb:274:in `require'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:36:in `initialize'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:10:in `run_and_exit'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:143:in `run_supported_command'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:125:in `run_command'
enterprisesearch_1  |   /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:112:in `run!'
enterprisesearch_1  |   bin/enterprise-search-internal:15:in `<main>'
elishowk_enterprisesearch_1 exited with code 1

I tried to google it, but couldn't find anything... Any idea? 🤔

(My env is MacOS 11.2.3 with the latest Docker version)

Recommend Projects